The California State University and MERLOT has designed a framework that will provide the accessibility community a systematic and easy-to-use method to evaluate the accessibility of course materials. This framework was designed to enable the accessibility community to be able to quickly ascertain useful and quality information concerning digital course materials so they will usable by all students irrespective of disability status. These methods can be used by publishers to improve the accessibility of their e-textbooks, and by faculty and students in their selection of e-textbooks.
The goals of the accessibility project were to (a) develop methods for assessing the accessibility of e-textbooks in different formats (e.g., E-Pub, HTML, PDF), (b) develop a scoring system for rating the accessibility of these e-textbooks, and (c) use the methodology and scoring system to evaluate the e-textbooks that are posted on the COOL4ed.org site.
The Accessibility Evaluation Methodology
The Summary Evaluation: Each “Accessibility Evaluation” begins with a summary and score (out of 10) for the overall etextbook accessibility score. The 10-point scale (with a score of 1 being the lowest score possible and a score of 10 being the highest) was used to provide users a snapshot of the accessibility evaluation.
The Detailed Report with Two Evaluation Formats (Assistive and Non-Assistive Technologies): The results of applying two evaluation methodologies (Assistive and Non-Assistive) are presented so that the user can determine the etextbook’s accessibility from the varied technologies available to users to navigate their digital textbooks.
- The “Non-Assistive Technologies (NAT) Evaluations” address readily available tools such as the keyboard and navigation typically found on our devices and are typically available to the general public
- The “Assistive Technologies (AT) Evaluations” address specialized tools such as Kurzweil and NVDA which are not typically available to the general public. Assistive Technologies includes but is not limited to the following:
- Accessibility features of desktop operating systems (e.g. high-contrast display themes, settings from the Keyboard and Mouse control panels)
- Accessibility-related software included with desktop operating systems (e.g. VoiceOver, Microsoft Narrator)
- Third-party accessibility software and hardware:
- Screen readers (e.g. JAWS, Window Eyes)
- Magnification software (e.g. ZoomText Magnifier/Reader, MAGIC Pro with Speech)
- Reading software for users with learning disabilities (e.g. Read and Write Gold, Kurzweil 3000)
- Refreshable Braille displays
The Accessibility Checkpoints
The following 15 checkpoints used in the accessibility evaluations are built upon the Section 508 technical standards and reviewed/approved by CAST, a nationally recognized organization with expertise in accessibility and UDL. The checkpoints have been organized and tailored to the unique characteristics of digital resources used in higher education courses. The scoring metric is a “pass/fail” qualifier, with a score of 7 and above being a “pass” and a score below 7 being a “fail”.
- Accessibility Documentation
- Text Access
- Text Adjustment
- Reading Layout
- Reading Order
- Structural Markup/Navigation
- Color and Contrast
- Science, Technology, Engineering and Math (STEM)
- Interactive Elements
Scoring the Textbooks
The evaluators developed and tested three different scoring techniques, and they determined that in the Summary section they would use a 10-point scale (with a score of 1 being the lowest score possible and a score of 10 being the highest) because it would allow users to quickly understand the results of the accessibility evaluation. They also determined a “pass/fail” qualifier for the Checkpoints section, with a score of 7 and above being a “pass” and a score below 7 being a “fail.”
Accessibility Evaluation Manuals
The evaluation framework was designed to provide the accessibility community a systematic and easy-to-use method to add their comments and evaluations concerning the accessibility of COOL4Ed resources. We evaluated the quality of the etextbooks with the intent of addressing the most effective tools, both in terms of capability and costs.
The following accessibility evaluation manuals address four different e-textbook formats (EPUB, HTML, PDF, and Word). Each section of the manual contains a short description of the checkpoint, what components of the e-textbook the evaluators should focus on, and how the evaluators should use the tools and materials provided to evaluate each checkpoint. The manuals were tested to ensure that they are understandable and easy to follow by novice users.
- Assistive Technologies (AT) Evaluation Manuals
- Non-Assistive Technologies (NAT) Evaluation Manuals
The accessibility evaluations were performed by the Center for Usability in Design and Accessibility (CUDA) at California State University, Long Beach in collaboration with the Chancellor's Office Accessible Technology Initiative (ATI); faculty and graduate students with expertise in human factors, usability, and accessibility performed the evaluations of over 150 free and open etextbooks. Project oversight by the director of CUDA, Gerry Hanley, Ph.D.
COOL4Ed.org has published the accessibility evaluation rubric and provides a detailed description of the methodology used to evaluate the accessibility of the etextbooks in COOL4Ed.
For More Information Contact:
Leslie Kennedy, Ed.D.
Director, Affordable Learning Solutions and Academic Technology Services
California State University, Chancellor’s Office.
(562) 951-4065 or firstname.lastname@example.org.