Integrating Accessibility Testing into the Development Process

Introduction To Usable Web Accessibility

Who Are The Accessibility Users?

Accessibility solutions benefit people with and without disabilities and are becoming increasingly available in standard computer hardware, mobile devices, operating systems, web browsers, and other tools. People in general access and navigate the Web in different ways, depending on their individual needs and preferences. People use different approaches to enter text and activate commands. For instance, some people do not use a mouse, keyboard, or both, while others use specific configurations for keyboard and mouse, or use alternative hardware or software altogether. Supporting an effective usage web experience by people with sensory, motor and cognitive impairments requires more than just accessible Web content. You must understand how people interact with the web, and the accessibility solutions that best accommodates their particular needs. This document will focus on the screen reader user experience, the accessibility web design guidelines, and the web component interoperability requirements.

How Do People Interact With The Internet?

Web Accessibility is the deficit gap between the user abilities and the system capabilities. The goal is to bridge the accessibility gap, through Inclusive Design, that will create the best possible interoperability of System components to achieve the desired User experience. Accessibility is a measurement of productivity, and productivity defines usability. Usability is concerned with effectiveness, efficiency and satisfaction through multiple means of representation, multiple means of expression, and multiple means of engagement. Usability is the ease of use and learnability of a human-made object, and includes methods of measuring the user experience. That is, Usability describes the quality of the user experience across the entire website. So, there is an implied essential partnership between the system components (operating platform, applications, assistive technology, and user knowledge), and project responsibility roles (Project management, Development, Architecture, Design, Content management, and testing), That must interact effectively for a good User experience.

Everyone that is browsing the web has a user agent. This is the software that acts as the bridge between the user and the internet. When your browser loads a web page, it identifies itself as an agent when it retrieves the content you have requested, and the browser sends a host of information about the device and network that you are on. This is a set of data that allows web developers to customize the web experience. That is, the browser must understand the Application Protocol Interface (API) building blocks and command structure of the web page, so as to display meaningful information on the user device and allow the user to interact with the page elements. There are many different browsers, each using a specific API structure and thus delivering a different user experience on different devices. However, for the most part, the browser API is designed for visual users and Mouse interaction, but some browsers have accommodated blind users by providing an accessibility interface structure for screen readers. A screen reader is a software application user agent tool that attempts to identify and interpret what is being displayed on the screen, whether a video monitor is present or not. This interpretation is then re-presented to the user with text-to-speech. Screen readers are a form of assistive technology (AT) potentially useful to people who are blind, visually impaired, illiterate or learning disabled. Thus, the screen reader must understand the API structure of the browser, so as to present the web page information in a meaningful synthesized speech or braille output format. Each of the many different screen readers use a specific API language to communicate with the many different browsers. The benefit in coding for the screen reader user agent API, is that you are at the same time coding for search engine optimization and small screen smart devices.

Web accessibility evaluation tools are software programs or online services that help you determine if web content meets accessibility guidelines. That is, they automatically evaluate a web page structure according to a specified accessibility standard. Screen readers, browsers and other user agents like video players, that adhere to the specified standard will result in a good user experience. However, the specified standard must be accepted and adopted by all user agent developers, to make the web page evaluation tool results meaningful.

Authoring tools are software and services that authors (web developers, designers, writers, etc.) use to produce web content. Although a web page may conform to the specified accessibility standard, it may not necessarily be usable by a screen reader user. The web page development techniques used to create the dynamic structure and device interaction behaviour, will determine how usable the web page content is to a screen reader user. to create an accessible Rich Internet Application developers and content managers must understand the interoperability of assistive technologies. Dynamic page rendering and operable functionality must be thoroughly tested before usability testing begins. Java scripts and widgets (JQuery, Dojo) must be robust. This is also refered to as User Acceptance testing, or Beta Testing, to ensure the code is behaving as expected and the website design is defect free before conducting usability testing. Screen readers use shortcut keys to navigate around web pages, but in every case they rely on the web page code being in place to support them. So it is important to use elements to convey semantic document structure and to use them according to specification.

Learn More About Web Accessibility

Web accessibility is essential for people with disabilities and useful for all. Learn about the impact of accessibility and the benefits for everyone in a variety of situations.

The Screen Reader User Experience

Screen Reader Navigation

many people with visual disabilities use screen readers which read aloud information on the screen such as text or image descriptions provided through alternative text. If you plan, format, and structure your internet content correctly in the beginning, it will ensure the content is not only accessible but is usable by a wide range of users, including people with disabilities and the aging population. It is important not to confuse screen reader assistive technology with self-voicing browsers and self-voicing operating system features.

Many screen reader users prefer to use multiple screen reader user agents to perform different tasks, because Some websites are designed to interact with specific browsers, while others use a very graphical interface. Understanding the WCAG standards and the expected behaviour of ARIA coded web pages, will help evaluate the usability of websites and which screen readers perform better. Accessibility awareness is a journey, as technologies and design techniques change daily. the understanding of The interoperability of website development best practices, website accessibility evaluation methods, and the user agent tools, are important for the creation of usability reports. That is, automated accessibility evaluation tools that are designed for specific user agent may return false issues, and screen reader users that do not understand the dynamic behaviour of web page content, or do not understand the screen reader user interactive controls, may present website developers with false accessibility issues. Using a defined template for reporting accessibility evaluation results will provide reporting consistency and a professional look. Also, providing a mechanism for feedback on the usefulness of the evaluation process and resulting report, may assist in ongoing identification of gaps in expertise, and contribute to long-term improvement in the quality of evaluations.

Using The JAWS Screen Reader

The Freedom Scientific JAWS screen reader can be a powerful tool in helping to evaluate the usability of a website. JAWS, a Microsoft Windows based screen reader assistive technology, presents Web pages using the JAWS Virtual Cursor. This allows users to read and navigate a Web page as though it were a text document. Users press the ARROW keys to read line by line, word by word, character by character, and so on. JAWS also provides Navigation Quick Keys, which are alpha-numeric keys that move the Virtual Cursor to features of the page such as links, headings, and form controls. In addition, users can press the TAB key to move between focusable elements on the page. Using the ARROW keys or Navigation Quick Keys to change the position of the Virtual Cursor does not change the actual focus point in the application. This means that even if JAWS reads the text of a given link on a Web page for example, that link doesn't necessarily have the keyboard focus. Conversely, pressing the TAB or SHIFT+TAB key to navigate moves the focus point and the Virtual Cursor follows the focus. The JAWS Navigation quick keys make it faster and easier to move around on a Web page and anywhere else the Virtual Cursor is active. These commands are all assigned to keys on the main part of the keyboard. A web page well structured with the HTML tags, make it possible for JAWS user to quickly navigate the page and gain an understanding of the page content.

JAWS uses four different Cursor Modes:

  1. The PC Cursor is linked to the keyboard functions of Windows and applications. This is the cursor that is used when typing information, moving through options in dialog boxes, and selecting options or icons. As you type information, the PC Cursor follows along with each key you press. pressing NUM PAD PLUS activates the PC Cursor.
  2. The JAWS Cursor is linked to mouse pointer functions in Windows and other applications. It is used to read information the PC Cursor cannot read, such as toolbar information. The mouse follows along with the JAWS Cursor when it is moved. press NUM PAD MINUS to activate the JAWS Cursor.
  3. The Touch Cursor enables you to navigate the Windows environment using a touch screen, found on many tablet computers running Windows 8 or later. To control JAWS from a touch screen, you will use one or more fingers to perform various gestures directly on the screen, such as tapping, flicking, and swiping. press SHIFT+NUM PAD PLUS to activate the Touch Cursor.
  4. The virtual PC Cursor mimics the functions of the PC Cursor, but is activated by default when entering an HTML document, such as a Web page on the Internet. The virtual PC Cursor speaks the number of elements on the page when it first opens. You can use the ARROW keys to navigate and read the document, or use Navigation Quick Keys to move to specific elements, such as paragraphs, tables, or headings. Press Insert+z keys to toggle the Virtual PC Cursor on and off, or Insert+z twice quickly for all opened Window applications.

Using The NVDA Screen Reader

NVDA (NonVisual Desktop Access) is a free, open source software, Microsoft Windows based assistive technology, screen reader which enables blind and vision impaired people to use computers. It reads the text on the screen in a computerised voice. You can control what is read to you by moving the cursor to the relevant area of text with a mouse or the arrows on your keyboard. NVDA can be installed on your computer, or a portable version can be use on an USB thumb drive. NVDA uses a virtual buffer concept common to all Windows screen readers on the market. NVDA takes the HTML of the page and converts it into a flat document with semantic information in the order the HTML appears in the browser source. As you navigate, NVDA will speak semantic information such as link, heading level 1 through heading level 6, button, images, and much more.

Using The Voiceover Screen Reader

VoiceOver is a screen reader program that comes on Mac computers, iPhones, iPads, and iPod touches. VoiceOver is not a standalone screen reader assistive technology. It is deeply integrated into the iOS operating system and all the built-in apps on iOS devices, and as developers update their apps to take advantage of the accessibility interfaces provided by Apple, their apps can start working with VoiceOver right away. VoiceOver currently only functions with the Safari, and with limited support Opera web browsers. It can also access most OS X native applications and functions. VoiceOver gives auditory descriptions of each onscreen element, and provides helpful hints along the way (whether you prefer using gestures, a keyboard, or a braille display), and it supports more than 30 languages, including multiple voice options.

VoiceOver is a sophisticated technology that provides many powerful features to users with disabilities. Although you do not need to become an expert VoiceOver user to test your app with it, you do need to know a handful of basic gestures. The Accessibility Inspector, available in the iOS Simulator, lets you simulate VoiceOver interactions, and examine the accessibility information that is available for the controls in your app. However, Accessibility Inspector does not have speech output, so it is a debugging tool rather than a testing tool. When it comes to testing, there is no substitute for using your app on an iOS device with VoiceOver turned on, or better still asking VoiceOver users to test your app. You can toggle VoiceOver on and off quickly by setting it to the triple-click setting in the Accessibility Settings. When VoiceOver is turned on you will need to use a different set of gestures. After you have turned VoiceOver on, you will notice that many familiar gestures have different effects. For example, a single tap causes VoiceOver to speak the selected item and a double tap activates the selected item. When an element is selected, VoiceOver draws a black rounded rectangle around it, which is called the VoiceOver cursor.

To simulate the experience a visually impaired user might have with your app, you can run it with the VoiceOver screen curtain in place. When you activate the screen curtain, VoiceOver turns off the device display so that no one can read. Testing with the display turned off obliges you to rely on the information VoiceOver speaks and removes the temptation to use your app as a sighted user would. To turn off the display while you use VoiceOver, triple-tap the screen with three fingers. To turn the display back on, perform the same gesture again.

Using The Windows Narrator Screen Reader

The Microsoft Windows Narrator screen reader is a screen-reading app built into the Windows operating system. Microsoft Narrator was written by Professor Paul Blenkhorn in 2000 and has been included with every version of Windows since that time. While Microsoft recommends that the visually impaired purchase a full-function screen reader for general computer use, Narrator is a useful piece of built-in accessibility software. Narrator is included with every copy of Microsoft Windows, providing a measure of access to Windows without the need to install additional software as long as the computer in use includes a sound card and speakers or headphones.

Narrator can assist a blind person in installing a full-function screen reader, assisting the user until their screen reader of choice is up and running. Because Narrator is a lightweight screen reader that requires minimal "hooks" into the operating system, Narrator can provide speech when a full-function screen reader might be unable to do so, such as during the process of updating hardware drivers. The Ease of Access settings in Windows are easy to discover, learn and use. Settings are grouped by ability (vision, hearing, and interaction), with the most frequently used settings listed first. To get directly to Narrator settings, press Windows logo key + Ctrl + N.

Using Self-Voicing Browser Screen Readers

A self-voicing application is an application that provides an aural interface without requiring a separate screen reader. Traditionally, talking web browsers had been specially created, but a more recent trend has seen the self-voicing capabilities added to mainstream web browsers with free add-ons. In 2004, Opera Software created a self-voicing and speech-recognition extension for the Windows version of their web browser, and in 2005 Charles L. Chen devised Fire Vox, an extension that adds speech capabilities to the Mozilla Firefox web browser on Mac, Windows, or Linux. A voice browser presents information aurally, using pre-recorded audio file playback or text-to-speech synthesis software. A voice browser obtains information using speech recognition and a keypad. As speech recognition and web technologies have matured, voice applications are deployed commercially in many industries and voice browsers are supplanting traditional proprietary interactive voice response (IVR) systems.

However, it is important to note that self-voicing browsers, despite vendor claims, cannot yet replace screen reader assistive technology, as they do not have the suffisticated interface required by blind users. The self-voicing browsers offer assistance to people who have trouble typing, moving a mouse, gesturing or reading a screen. Many people with disabilities have difficulty using computers and handheld devices. Self-voicing browsers allow people with limited dexterity to overcome barriers. Such as, those with quadriplegia, Parkinson's disease, multiple sclerosis, Cerebral Palsy, and other conditions that make it challenging to use touchscreen smartphones and tablets. The eSSENTIAL Accessibility self-voicing browsing tool is a cloud based self-voicing browser service that can be integrated into an organizations website. Another self-voicing browsing service is The AudioEye Personalization Platform, which is an all-inclusive SaaS Platform built to support the compliance auditing process end-to-end. Another popular tool is the BrowseAloud, which allows you to change the text, size and colours of the text, as well as having it read out loud to you.

Screen Reader Tips and Resources

  1. JAWS Basic Set Up and User Preferences

    • Press Insert+j keys to open the Options menu to set the Basics and Voices settings.
    • JAWS can present HTML page information according to the user preferences. User preferences can be set in the JAWS Quick Settings dialog (Insert+v), or from the JAWS Manager (Insert+f2 Settings Center)
    • JAWS can be configured to use many different Speech and Sound Schemes in the Settings Center (Press Insert+f2, select Settings Center, and search for "scheme").
  2. Some JAWS Keyboard Commands

    • Press Insert+1 to toggle keyboard help on and off.
    • Press Insert+f1 to get application related help.
    • Press Insert+f1 twice quickly to get application related help topics.
    • Press Insert+h to get application hotkeys.
    • Press Insert+w to get Window key commands.
    • Press Insert+j followed by the letter h key and then down arrow to select Freedom Scientific Help information.
    • View the JAWS Navigation Quick Key Manager: Press Insert+f2, select Navigation quick keys manager.
    • Press Insert+f1 to display the type and number of HTML elements on the page.
    • Press Insert+Shift+f1 to display HTML tag elements and attributes.
    • Press Insert+f to display the current cursor position Font type and point size.
    • Press Insert+5 to display the current cursor position colour.
    • Press Insert+Space, followed by the letter h, to open the speech history, and press Escape to close it.
    • Press Insert+Ctrl+s to select a Voice Profile.
    • Press Insert+Alt+s to select a speech scheme profile.
  3. More Screen Reader Resources

Accessibility Testing Guidelines

Accessibility Conformance Standards and Reporting

The World Wide Web Consortium (W3C) is an international community where Member organizations work together to develop Web standards. Tim Berners-Lee invented the World Wide Web in 1989, and founded the W3C organization. The W3C mission is to lead the Web to its full potential, and the Web Accessibility Initiative (WAI) develops a set of guidelines that are internationally recognized as the standard for web accessibility, which include the WCAG standard. The guidelines and Success Criteria are organized around four principles. Under each of the principles are guidelines and Success Criteria that help to address these principles for people with disabilities. One of the key objectives of the guidelines is to ensure that content is directly accessible to as many people as possible, and capable of being re-presented in different forms to match different peoples' sensory, physical and cognitive abilities. Under each guideline, there are Success Criteria that describe specifically what must be achieved in order to conform to WCAG at three levels of conformance: A, AA, AAA. All WCAG Success Criteria are written as testable criteria for objectively determining if content satisfies the Success Criteria. While some of the testing can be automated using software evaluation programs, others require human testers for part or all of the test. Although content may satisfy the Success Criteria, the content may not always be usable by people with a wide variety of disabilities. Get started with First Review of Web Accessibility: Web Accessibility Initiative (WAI)

WCAG Principles

  1. Perceivable: Information and user interface components must be presentable to users in ways they can perceive. This means that users must be able to perceive the information being presented. User agents, like screen readers, require clearly defined HTML elements within a structured web page. The ARIA (Accessibility Rich Internet Application) Landmarks and a hierarchy of Headers should be used to define page regions and content context. The Banner, Navigation panel, Main section, and Footer are visually perceivable on a standard computer screen, but not necessarily on a screen reader device.
  2. Operable: User interface components and navigation must be operable. This means that users must be able to operate the interface. All web page elements must be operable by a keyboard, speech input, and other non-mouse devices. Some of the Java scripts may not be keyboard accessible, and preventing non-mouse users from performing some functions.
  3. Understandable: Information and the operation of user interface must be understandable. This means that users must be able to understand the information as well as the operation of the user interface. Page Titles must be unique and meaningful. Links and Buttons must have concise and clearly marked text labels. Images must have descriptive alternative text. The page foreground and background, and Icons, must have contrasting colours for low vision users. The web page must have clearly defined user instructions, and a separation of information content.
  4. Robust: Content must be robust enough that it can be interpreted reliably by a wide variety of user agents, including assistive technologies. This means that users must be able to access the content as technologies advance. To deliver a desirable user experience, there must be a separation between web page design and user content. The web page may not render as expected in all browsers, and will not perform as expected in differing user agents. A design utilizing style sheets and Java Script widgets may improve the robustness. Note, the Accessibility Rich Internet Application (ARIA) code should only be used on a webpage if the native HTML code cannot implement the desired effect. ARIA code will not have any effect on older browsers.

Using ARIA To Enhanced Usability

The WAI Accessible Rich Internet Applications (ARIA) Suite, defines a way to make Web content and Web applications more accessible to people with disabilities. It especially helps with dynamic content and advanced user interface controls developed with Ajax, HTML, JavaScript, and related technologies. Currently certain functionality used in Web sites is not available to some users with disabilities, especially people who rely on screen readers and people who cannot use a mouse. WAI-ARIA addresses these accessibility challenges. ARIA has created an accessibility bridge between the standard HTML specification and the dynamic features of complex website applications of today. The ARIA specification provides a framework to describe features for user interaction, and defines how information can be provided to assistive technologies like screen readers. Assistive Technologies use this to provide services beyond those offered by the web browser to facilitate user interaction with web content by people with disabilities. ARIA user interaction controls enable assistive technologies to provide a better user experience. WAI-ARIA techniques apply to widgets such as buttons, drop-down lists, calendar functions, tree controls, expandable menus, and much more. One of the greatest challenges in learning to implement and test ARIA properly, is that ARIA is an invisible technology. Sighted observers cannot see where ARIA is applied, nor when issues are caused as a result of its use. The accessible use of ARIA learning can only be achieved through experimentation and experiential learning. ARIA can be viewed as the audio representation of CSS, and like CSS the user experience will depend upon which browser they are using, which operating system they are using, and the current level of combined support between the operating system, the browser, and the assistive technology, all of which differ significantly when combined in different variations.

In HTML, only links and form elements can receive keyboard focus. This means that as you tab through a page, the browser stops the focus only on these types of elements. With scripting, however, you can add mouse interactivity to nearly any element. This means you can make normal page elements, such as a paragraph or span, interactive and responsive to the mouse, such as making plain text display and behave like a button. However, the functionality of these non-focusable elements cannot be made accessible to screen reader and keyboard-only users. ARIA provides mechanisms for non-focusable elements to receive focus through the tabindex property. By setting the tabindex=0, an element will be placed in the tab order of the document, So when the user tabs to the element the browser will stop and set focus to the element. This allows additional functionality and interactivity to be applied to the element, such as triggering functionality when the element receives keyboard focus or when the user presses a key while the element has focus. By establishing a standard set of roles, properties, and values, ARIA allows developers to address these accessibility issues with relative ease. ARIA Roles are defined and described by their characteristics. Characteristics define the structural function of a role, such as what a role is, concepts behind it, and what instances the role can or must contain. In the case of widgets this also includes how it interacts with the user agent based on HTML mapping. User agents MUST map all supported states and properties for the role to an accessibility API. Content authors MAY provide values for supported states and properties, but need not in some cases where default values are sufficient. In order to reflect the web page content, user agents must map the role attribute to the appropriate value in the implemented accessibility API, and user agents must update the mapping when the role attribute changes. The Role text Name comes from values provided by the developer in explicit markup features. This can be done by using aria-label attribute, aria-labelledby attribute, or the host language labeling mechanism, such as the alt or title attributes in HTML, with HTML title attribute having the lowest precedence for specifying a text alternative. Also, the Role text name can be specified by the text value of the element node, if higher priority attribute features are not provided. Note, there are different rules provided for several different types of nodes and combinations of markup. Priority is defined by the text alternative computation algorithm. Read the WebAIM article: To ARIA! The Cause of, and Solution to, All Our Accessibility Problems.

ARIA Role Classification

  1. Landmark Roles: These roles are regions of the page intended as navigational landmarks.
  2. Document Structure Roles: These roles describe structures that organize content in a page. Note, document structures are not usually interactive.
  3. Widget Roles: These roles act as standalone user interface widgets or as part of larger, composite widgets.
  4. Abstract Roles: Abstract roles are the foundation upon which all other ARIA roles are built. Composite user interface widgets typically act as containers that manage other, contained widgets.

ARIA Roles And Properties

  • Roles to describe the type of widget presented, such as menu, treeitem, slider, and progress meters.
  • Roles to describe the structure of the Web page, such as regions, headings, tables, grids, and lists.
  • Properties to describe the state widgets are in, such as Checked for a check box, Expand or Collapsed state, selection On or Off, and Has Popup for a menu.
  • Properties to define live regions of a page that are likely to get updates (such as stock quotes), as well as an interruption policy for those updates (for example, critical updates may be presented in an alert dialog box, and incidental updates occur within the page).
  • Properties for drag-and-drop that describe drag sources and drop targets.
  • A way to provide keyboard navigation for the Web objects and events, such as those mentioned above. see: Web applications and ARIA FAQ Accessibility - Mozilla Developer Network (MDN)

Testing for Accessibility and Usability

The accessibility of a web site is important for attracting customers, but the usability of the web site will determine the level of success in which customers engage your organization. Accessibility and usability are closely related, and the inclusive design model should integrate their goals, approaches, and guidelines to ensure web site development reflects expectations. Combining accessibility standards and usability processes with real people ensures that web design is technically and functionally usable by people with disabilities.

Accessibility means enabling people with diverse abilities and methods of access to effectively use your interface. The aim of accessibility is to remove barriers for perceiving, understanding and navigating your interface and ensure that nobody is excluded. Usability, on the other hand, is more to do with designing a user-friendly approach. It is concerned with how people interact with a website's interface. That is, is it easy to do the things people want to do? Can people quickly find what they want without help? Do people make lots of mistakes using a website? Do people enjoy using the website? In general, usable websites benefit everyone. Whilst usability implies accessibility, the contrary is not necessarily true.

If designers, developers, and project managers approach accessibility as a checklist of meeting accessibility standards, the focus is only on the technical aspects of accessibility, and the human interaction aspect is often lost. In general, designers have to consider two different groups of users when aiming to make their interfaces accessible for all. people with differing technical abilities and people with differing physical abilities. Understanding the technical abilities of your users entails considering what devices they are using, where they will be accessing your content and how technically literate they are. Not only do designers have to keep user technical limitations in mind, but they also have to consider users with disabilities including visual, auditory, physical, speech and neurological disabilities.

So what does good usability look like? You will generally find that good usability goes unnoticed. In other words, a highly usable website means that your users are getting around so easily that they don't even know how good they have it. The Internet has the potential to broaden the lives and increase the independence of people with disabilities. For people who can be physically as well as socially isolated, access to the Internet can offer information about social interaction, cultural activities, employment opportunities, and consumer goods. But, as statistics demonstrate, not many people with disabilities are able to take advantage of these possibilities, in large part because their needs have not been addressed by the web design community.

Using Automated Tools For Functional Testing

Accessibility evaluation tools perform web page automated checks, but each is designed for a specific target audience based on a specific conformance standard. So, it is important to choose an appropriate evaluation tool for the intended reporting purpose. Consider the standards and guidelines used by the different accessibility evaluation tools. There are currently two standards and guidelines most commonly used: the Web Content Accessibility Guidelines and the Section 508 of the United States Rehabilitation Act. Some tools are available at a web site where they are able to evaluate the content of a page quickly and easily without downloading or installing an application. Other options include tools that are created as extensions to a browser, and some tools require installation on your hard drive or server. Some tools are simple and limited and evaluate just one page at a time. While other tools are very detail oriented that are able to examine large sites and check for a variety of errors. Many tools can only perform an evaluation and may be free to use, but many commercially available tools provide repair guidance in addition to the evaluation process. Accessibility report styles differ widely depending on the target audience, such as web designers and developers or executives and management, but in general tend to be very lengthy and challenging to decipher. Although evaluation tools can identify accessibility issues, they cannot determine if a product complies with expected conformance levels and is usable by assistive technologies. That is, tools can identify images that are missing alt text, but they cannot determine how appropriate the text is in describing the image.

Performing Usability Testing

No automated evaluation tool can tell you if your site is accessible, or even compliant. Human testing is always necessary because accessibility is about the human experience. Accessibility evaluation is often limited to assessing conformance to accessibility standards, according to legal requirements or a defined business best practice. However, when the focus is only on the technical aspects of accessibility, the human interaction aspect can be lost. Many of the web page accessibility checks require human judgement and must be evaluated manually using different techniques. In some cases evaluation tools are prone to producing false or misleading results, and should not be used to determine conformance levels of accessibility. Web accessibility evaluation tools can not determine the accessibility of Web sites, but can only assist in doing so. Usability evaluation methods can assess usable accessibility to ensure that accessibility solutions are usable by people with disabilities. Inclusion of people with disabilities in a collaborative group can contribute to a better understanding of accessibility issues within the organization. Websites often have text that is difficult to read, controls that are difficult to click, or audio and videos that are difficult to hear. Fortunately in many cases the user can customize their computer to improve the web browsing experience. This includes customization options in the operating system, in software such as web browsers and media players, and sometimes for hardware devices such as any external loud speakers or microphones. Sometimes changing the user' web browser, using additional software or hardware, or otherwise customizing the computer can further improve the accessibility user experience.

Testing and Reporting Strategy

Today web technologies (operating systems, browsers and assistive technologies) all work together to extract accessibility information from a web interface and present it to the user. If appropriate content semantics are not available, then assistive technologies will use old and unreliable techniques to make the interface usable. Making this information available through the Accessibility Application Programming Interface (API) is more efficient and less prone to error than relying on assistive technologies to create an off-screen model or guess at the information they need. It is now possible to make an interface developed with well-written HTML, CSS and JavaScript very accessible and usable for assistive technology users. There are several Accessibility API Mapping (AAM) standards that the tester must be aware of. These documents describe how internet browsers map HTML content to the Accessibility Application Programming Interfaces of assistive technologies.
  1. Microsoft Active Accessibility (MSAA): an Accessibility API devised by Microsoft for Windows based applications to interface with assistive technologies.
  2. IAccessible2 (IA2): IAccessible2 is a new Accessibility API, which complements the Microsoft earlier work on MSAA, to fill critical Accessibility API gaps in the MSAA.
  3. Assistive Technology Service Provider Interface (AT-SPI): an Accessibility API devised by Sun Microsystems so that accessibility aids can track what's going on inside the user interface of any software package that supports it.
  4. Universal Access (UA): Universal Access (UA) is the Apple Accessibility API framework for the iOS platform.

Accessibility Interoperability Testing

Accessibility API Process

Typically, user interfaces are represented as a hierarchical tree. For example, an application window would contain several objects, the first of which might be a menu bar. The menu bar would contain a number of menus, each of which contains a number of menu items, and so on. The Accessibility API describes the object relationships to other objects, to provide context. Other features such as information about text formatting, applicable headers for content sections or table cells and things such as event notifications, are also available in the Accessibility API.
  1. A web developer writes some HTML code.
  2. That HTML code is retrieved and parsed by a user agent browser.
  3. The user agent browser uses the appropriate Accessibility API Mapping standard to create, and maintain, an Accessibility Tree Document Object Model (DOM) representation with only HTML information that is relevant to the assistive technology screen reader.
  4. The Accessibility Tree is exposed via the platform Accessibility API.
  5. The assistive technology screen reader then interacts with the Accessibility API to retrieve information about the web page.
  6. The assistive technology screen reader reads the content to the end user via a refreshable braille display and/or synthesized speech output.
  7. A Key To Web Accessibility: By Leonie Watson & Chaals McCathie Nevile, March 2015

Accessibility API Concepts

Assistive technologies can make standard method calls to the operating system to get information about the objects on the screen. In browsers, the platform Accessibility API is used both to make information about the browser itself available to assistive technologies and to expose information about the currently rendered content. Browsers typically support one or more of the available Accessibility APIs for the platform they are running on. For example, on Windows, Firefox, Chrome, Opera and Yandex, support MSAA, IAccessible and IAccessible2. While Internet Explorer supports MSAA, IAccessible and UIAExpress. Safari and Chrome support NSAccessibility on OSX and UIAccessibility on iOS. ChromeVox uses DOM access exclusively to provide web content information to assistive technologies, and does not use the Chrome Accessibility API. Typically, browsers use the HTML DOM, along with further information derived from CSS, to generate an accessibility tree hierarchy of the content it is displaying, and it passes that information to the platform Accessibility API. Information such as the role, name and state of each object in the content, as well as how it relates to other objects in the content, can then be queried by assistive technologies. While each operating system and vendor Accessibility API is different, there are some concepts all of them share.
  1. The tree, which models the entire interface as a tree of objects, exposed to assistive technology via the Accessibility API.
  2. The events, which let assistive technology know that a part of the tree has changed somehow.
  3. The actions, which come from assistive technology to ask the interface to change.

Evaluating The User Experience

Evaluating the accessibility of Web content for people with disabilities requires diverse kinds of expertise and perspectives. Comprehensive and effective evaluations require testers with an understanding of Web technologies, evaluation tools, barriers that people with disabilities experience, assistive technologies and approaches that people with disabilities use, and accessibility guidelines and techniques. The scope of the accessibility evaluation will depend upon the skill level of the website developers in understanding standards, techniques, and user interaction. A standards review in the User-Centered Design process assesses whether a product conforms to specified interface design standard. A best practice inclusion model requires standards that are adopted throughout the entire organization, so as to achieve a good user experience through the interoperability of policies and techniques. To get started view the W3C-WAI Accessibility Evaluation Reporting Template

The Inclusion Baseline

  1. Define the standards and conformance guidelines for digital communications. The WCAG standard, conformance AA, is the globally accepted best practice.
  2. Define Accessibility law requirements for which digital communications must comply. This is generally governed by legislation through various level of government; Such as The Ontario AODA standards, and The U.S. Section508 standards.
  3. Define evaluation tasks. This can be performed by automated evaluation tools for functional testing, and end-user focus groups for usability testing.
  4. Define a baseline of web technologies. The goal is to make digital communications accessible to as many people as possible, but by defining a baseline of user agents (browsers, screen readers, media players, Etc), and system platforms (Windows, iOS, Android, mobile devices, Etc), will identify the scope of expectations.
  5. Define testing methodology. The evaluation results will depend upon the tools used, level of expertese, and user involvement.
  6. Define test procedures. Testing digital communications across hardware devices, software applications, levels of user experience, and developer expertese, will require standard Persona scripts, assessing specific web pages, and assessing specific dynamic functions.
  7. Define the report style. Generating an accessibility evaluation report can be challenging and difficult for others to understand. Depending upon the report audience, consider documents and videos that comply with accessibility standards, with remediation recommendations.
  8. Define the delivery process. To meet expectations the accessibility evaluation results must be delivered in a format and in a way it can be understood and acted upon. Consider Executive summaries, management implementation overviews, and developer coding techniques.

Issue Type

Identify the interoperability usability type of the accessibility challenge:
  1. The developer did not markup/code the web page properly, or
  2. The browser or media player isn't handling the markup properly, or
  3. The user's Assistive Technology (AT) isn't handling the markup properly, or
  4. The user doesn't know how to use the browser, media player, and user agent keyboard access features, or
  5. The page is poorly designed and it is a general usability problem for all users, including those without disabilities.

Issue Severity

To minimize development costs and Testing cycles identify the severity of the accessibility challenge and the remediation responsibility:
  1. Critical: Must fix to allow even the most basic use of the application. User with a disability cannot complete a task, and no alternate means is provided to complete that task. The issue is a violation of the Web Accessibility Checklist.
  2. High: Must fix in order to meet accessibility standards and allow full use of the system. User with a disability will likely not be able to easily complete a task, and no alternate means is provided to complete the task. The issue is a violation of the Web Accessibility Checklist.
  3. Medium: Should fix to allow productive, accessible use of the application. User with a disability will likely be able to complete a task, but the issue prevents the user from completing the task efficiently. The issue may or may not be a violation of the Web Accessibility Checklist.
  4. Low: User with a disability will be able to complete a task, but the issue may cause confusion to the user, and should be resolved. The issue may not be a violation of the Web Accessibility Checklist. These may be functionality bugs that may effect all users.

Issue Rating

Identify the overall usability website rating and the accessibility conformance:
  1. Effectiveness: Can users complete tasks and achieve their goals on the web application?
  2. Efficiency: How much effort and time do users require to complete the task?
  3. Satisfaction: Do users find the website easy to use and learn?

Additional Resources

Developer ARIA Coding Resources

Developer API Resources

Abbreviated Accessibility Checklist

  • WCAG1.1.1: Alternative Image text
    All non-text content that is presented to the user has a text alternative that serves the equivalent purpose. Such as, pictures and charts have alternative text descriptions.
  • WCAG1.3.1: Content relationships
    The web page should be divided into perceivable block areas, which contain semantically associated information called Regions. Each region can be divided into logical subregions as needed. For users with cognitive and learning disabilities the landmark information could be used to expand and collapse these regions of your page to aid in simplifying the user experience by allowing the user to manage the amount of information processed at any one time.
  • WCAG1.4.1: Use of colour
    Colour alone is not used to convey hierarchy, functionality and content. For people with visual disabilities such as colour blindness, relying solely on colour to convey hierarchy, functionality and content means they will not be able to use the product. Using more than three colours may make content understanding difficult. sometimes, for specific cases, more than three colours can be used, but this is not a fixed rule.
  • WCAG1.4.3: Sufficient background and text contrast
    Foreground elements are easily distinguished from the background. Clear distinction facilitates navigation, brings more attention to buttons and increases usability.
  • WCAG1.4.8: Visual presentation
    There should be clear indicators as to what is content and what are controls. These indicators can be size, colours, font etc. By using different styles or families, user will not be confused and will easily identify what can be interacted with.
  • WCAG2.1.1: Keyboard Access
    The intent of this Success Criterion is to ensure that, wherever possible, content can be operated through a keyboard or keyboard interface (so an alternate keyboard can be used). When content can be operated through a keyboard or alternate keyboard, it is operable by people with no vision (who cannot use devices such as mice that require eye-hand coordination) as well as by people who must use alternate keyboards or input devices that act as keyboard emulators. Keyboard emulators include speech input software, sip-and-puff software, on-screen keyboards, scanning software and a variety of assistive technologies and alternate keyboards.
  • WCAG2.4.1: Skip content
    People who use magnifiers to view a web site find skip links useful. Often a person using a magnifier will only see a small part of the screen. Often it is not obvious where the content is from this small part of the screen. This is in contrast to a sighted user's ability to ignore the repeated material either by focusing on the center of the screen (where main content usually appears) or a mouse user's ability to select a link with a single mouse click rather than encountering every link or form control that comes before the item they want.
  • WCAG2.4.2: Page Title
    Every web page should have its own, unique Title element that identifies the page content and its website orientation.
  • WCAG2.4.4: Link purpose
    Ensure Link Text labels Make Sense. The intent of this Success Criterion is to help users understand the purpose of each link so they can decide whether they want to follow the link. Whenever possible, provide link text that identifies the purpose of the link without needing additional context. Assistive technology has the ability to provide users with a list of links that are on the Web page. Link text that is as meaningful as possible will aid users who want to choose from this list of links. Meaningful link text also helps those who wish to tab from link to link. Meaningful links help users choose which links to follow without requiring complicated strategies to understand the page.
  • WCAG2.4.5: Breadcrumb
    Breadcrumbs are an additional navigation tool that assists both the general public and people with disabilities to navigate through a large or complex site. It is especially important where other navigation mechanisms, such as the Back button, have been broken. Breadcrumbs should provide the hierarchical position not the chronological pathway in the site. For instance, even if a user came to a particular part of the site through inline links, the breadcrumbs should provide the navigational pathway to that page.
  • WCAG2.4.6: Logical Heading hierarchy
    It is important that headings are nested properly in order to convey the structure and hierarchy of the page. It is important not to skip heading levels (Example, a H4 should not follow an H2). Screen reader users scan a web page using a variety of features, such as links, form controls and headings. Most screen readers can pull out the headings, and present them to the screen reader user in hierarchical order. This can provide a great amount of information to users and help them navigate through the page effectively.
  • WCAG2.4.7: Visual focus
    Make Sure Links are Recognizable. The purpose of this success criterion is to help a person know which element has the keyboard focus.
  • WCAG3.1.1: Page language
    Both assistive technologies and conventional user agents can render text more accurately when the language of the Web page is identified. Screen readers can load the correct pronunciation rules. Visual browsers can display characters and scripts correctly. Media players can show captions correctly. As a result, users with disabilities will be better able to understand the content.
  • WCAG3.2.3: Consistent navigation
    The way user navigates through the website directly influences whether they can achieve their goals irrespective of the page they are on. That is, navigational mechanisms that are repeated on multiple Web pages within a set of Web pages occur in the same relative order each time they are repeated.
  • WCAG3.3.1: Easy error recovery
    Users may perform actions that lead them to undesired results. Allowing the users to go back and given the opportunity to try again will reduce frustration.
  • WCAG3.3.2: Form fields
    Ensure Forms have explicit labels and/or titles. Instructions or labels that identify the controls and data fields in a form are clear so that users know what input data is expected. Instructions or labels may also specify data formats for fields, like dates and times.
  • WCAG4.1.1: Invalid HTML Parsing
    HTML Elements have start and end tags, are nested according to their specifications, do not contain duplicate attributes, and element Ids are unique. The intent of this Success Criterion is to ensure that user agents, including assistive technologies, can accurately interpret and parse content. If the content cannot be parsed into a data structure, then different user agents may present it differently or be completely unable to parse it.
  • Learn more about WCAG success criteria.