Making Accessibility Testing Suck Less:

An Intro to Pa11y

Jennifer Wadella

Thanks to all our sponsors!

Jennifer Wadella


  • JavaScript Developer
  • Community Organizer
  • Fairly Decent Programmer

Accessibility - why should I care?

Visually Impaired Americans

Hearing Impaired Americans

Physically Impaired Americans

Impaired Americans

Data source: National Center for Health Statistics, Summary Health Statistics: National Health Interview Survey, 2015 ,


Accessibility (a11y) is a measure of a computer system's accessibility is to all people, including those with disabilities or impairments. It concerns both software and hardware and how they are configured in order to enable a disabled or impaired person to use that computer system successfully.

Setting the Stage for Accessibility Testing

Issues to be aware of:

Visual Impairments

  • blindness
  • color blindness(Deuteranopia, Protanopia, Tritanopia)

Auditory Impairments

  • Deafness or partial deafness
  • Needing to rely on subtitles & text

Motor Impairments

  • Physical impairments - ex. loss of limbs
  • Age - affecting coordination
  • Diseaes - affecting motor control
  • Struggles using mouse or keyboard

Other Considerations

  • Mobile devices
  • Slow internet connectivity
  • Enlarged/zoomed in text

Just pretend Det. Lincoln Rhyme needs to be able to navigate your website to solve a homicide

Accessibility Checklist

  • Landmark Roles
  • Language Attribute
  • Semantic Headings
  • Links - :focus state and recognizable
  • Image alt text
  • Forms - labels, placeholdes, and fieldsets
  • Audio/video - provide transcripts/subtitles
  • Color & Contrast
  • Screen Reading/Keyboard Testing

Let's review some standards!

WCAG 2.0 Guidelines

§ 1194.22 Web-based intranet and internet information and applications.


  1. Provide text alternatives for any non-text content so that it can be changed into other forms people need, such as large print, braille, speech, symbols or simpler language.
  2. Provide alternatives for time-based media.
  3. Create content that can be presented in different ways (for example simpler layout) without losing information or structure.
  4. Make it easier for users to see and hear content including separating foreground from background.


  1. Make all functionality available from a keyboard.
  2. Provide users enough time to read and use content.
  3. Do not design content in a way that is known to cause seizures.
  4. Provide ways to help users navigate, find content, and determine where they are.


  1. Make text content readable and understandable.
  2. Make Web pages appear and operate in predictable ways.
  3. Help users avoid and correct mistakes.


  1. Maximize compatibility with current and future user agents, including assistive technologies.


confused crypto kitty
  • Level A – the most basic web accessibility features
  • Level AA – deals with the biggest and most common barriers for disabled users
  • Level AAA – the highest (and most complex) level of web accessibility

Section 508 Standards

Section 508 is the amendment to the United States Workforce Rehabilitation Act of 1973, a federal law mandating that all electronic and information technology developed, procured, maintained, or used by the federal government be accessible to people with disabilities.

  • Published December 21, 2000
  • Part of the Federal Acquisition Regulation (FAR) - address access for people with physical, sensory, or cognitive disabilities
  • Updated on January 17th, 2017 - this integrated WCAG 2.0
  • For all other non-procured ICT, federal agencies and contractors must comply with the updated 508 Standards beginning on January 18, 2018.
  1. A text equivalent for every non-text element shall be provided (e.g., via “alt”, “longdesc”, or in element content).
  2. Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation.
  3. Web pages shall be designed so that all information conveyed with color is also available without color, for example from context or markup.
  4. Documents shall be organized so they are readable without requiring an associated style sheet.
  5. Redundant text links shall be provided for each active region of a server-side image map.
  6. Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape.
  7. Row and column headers shall be identified for data tables.
  8. Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or column headers.
  1. Frames shall be titled with text that facilitates frame identification and navigation.
  2. Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.
  3. A text-only page, with equivalent information or functionality, shall be provided to make a web site comply with the provisions of this part, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes.
  4. When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with functional text that can be read by assistive technology.
  5. When a web page requires that an applet, plug-in or other application be present on the client system to interpret page content, the page must provide a link to a plug-in or applet that complies with §1194.21(a) through (l).
  6. When electronic forms are designed to be completed on-line, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues.
  7. A method shall be provided that permits users to skip repetitive navigation links.
  8. When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required.

Major Changes- Final Update

  • Restructuring provisions by functionality instead of product type due to the increasingly multi-functional capabilities of ICT;
  • Incorporating the Web Content Accessibility Guidelines (WCAG) 2.0 by reference and applying Level A and Level AA Success Criteria and Conformance Requirements to websites, as well as to non-web electronic documents and software;
  • Specifying the types of non-public facing electronic content that must comply;
  • Requiring that operating systems provide certain accessibility features;
  • Clarifying that software and operating systems must interoperate with assistive technology (such as screen magnification software and refreshable braille displays);
  • Addressing access for people with cognitive, language, and learning disabilities; and
  • Harmonizing the requirements with international standards.

Porg panic

How Do I Keep this All in Mind While Coding?

Not the hero we need

pa11y logo

but the hero we deserve

Pa11y is your automated accessibility testing pal. It runs HTML CodeSniffer from the command line for programmatic accessibility reporting.

Let's test some websites!

Rupaul opera glasses

pa11y cli is fun ...


							pa11y --reporter html > codemash-report.html

							pa11y --ignore "warning;notice" --reporter html > codemash-report.html

							pa11y --standard Section508


but not super useful in a typical workflow

From the cli

  • Run an accessibility test against a url
  • Use a reporter to output results to a file (csv,tsv,html,json,markdown)
  • Choose the accessibility ruleset
  • Choose to ignore warnings and notices


CI runs accessibility tests against multiple URLs and reports on any issues. This is best used during automated testing of your application and can act as a gatekeeper to stop a11y issues from making it to live.


							  "defaults": {
							    "standard": "WCAG2AAA"
							  "urls": [
							      "url": "http://localhost:4200/ethereum101",
							      "actions": [
							        "click element #myButton",
							        "wait for element modal-container to be hidden"

Define your fail threshold

							pa11y-ci --threshold 10

How many errors before "failure" is determined

Using the JS Interface

							var pa11y = require('pa11y');

							var test = pa11y({
							  //config options
							});'http://localhost:4200/', function(error, result) {
							  if (error) {
							    return console.error(error.message);

Result output

							  code: 'WCAG2AA.Principle1.Guideline1_1.1_1_1.H30.2',
							  context: '',
							  message: 'Img element is the only content of the link, but is missing alt text. The alt text should describe the purpose of the link.',
							  selector: 'html > body > p:nth-child(1) > a',
							  type: 'error',
							  typeCode: 1


							  actions: [
							    'click element #tab-1',
							    'wait for element #tab-1-content to be visible',
							    'set field #fullname to John Doe',
							    'check field #terms-and-conditions',
							    'uncheck field #subscribe-to-marketing',
							    'wait for #modal to be visible',
							    'wait for fragment to be #page-2',
							    'wait for path to not be /login'

before Script

							  beforeScript: function(page, options, next) {
							    // Make changes to the page
							    // When finished call next to continue running Pa11y tests

hide Elements

							  hideElements: '.advert, #modal, div[aria-role=presentation]'


							  ignore: [


							  log: {
							    debug: console.log.bind(console),
							    error: console.error.bind(console),


							  page: {
							    headers: {
							      Cookie: 'foo=bar'
							    settings: {
							      loadImages: false,
							      userName: 'nature',
							      password: 'say the magic word'
							    viewport: {
							      width: 1024,
							      height: 768


							  rules: [
							  standard: 'WCAG2AAA'

screen Capture

							  screenCapture: __dirname + '/my-screen-capture.png'


							  timeout: 500,
							  wait: 500

verify Page

							  verifyPage: 'All About Cryptokitties'

Let's test some more websites!

Rupaul clapping

Example Code Here:

Ugh, that seems like a lot of work

Rupaul doesn't have time for that

pa11y Dashboard


  • we wrote a basic test
  • we output results to html file
  • we created a custom report html file
  • we created a custom dashboard
  • we used actions to aid tests
  • we ignored certain elements


My Angular app isn't throwing any errors?

Your code is perfect! JK. It's probably not loading in phantomjs. Take a look at your polyfills:

							import 'core-js/es6/symbol';
							import 'core-js/es6/object';
							import 'core-js/es6/string';
							import 'core-js/es6/array';


My layout looks broken in screencaptures?

Are you using flexbox? PhantomJS doesn't support flex = (

  • Workaround 1: Add prefixes
  • Workaround 2: Use a polyfill + PostCSS processor
  • Workaround 3: Use the pa11y 5.0 beta - chrome likes flexbox!


My single page app pages aren't loading in the pa11y 5.0 beta!

Puppeteer broke window.pushState support in the current release. You can use actions to click nav items as a temporary workaround.

coming to a repo near you:

pa11y 5.0

  • Switching from PhantomJS to Headless Chrome
  • Only supports Node.js v8.0.0 and higher
  • Ignores warnings and notices by default
  • Promise-based, and uses async/await internally.
    This means the API has changed significantly.
  • No longer need to create a separate test function,
    now Pa11y is just one function: pa11y(url, options)
  • 4.0 reporters have been removed
  • Default viewport dimensions changing from 1024x768 to 1280x1024
  • Configuration option changes - mostly related to removing PhantomJS

Arguing the business side

  • Minimal effort to implement
  • Immediate improvements available
  • Audience retention
  • Build customer loyalty
  • Help improve SEO
  • Avoid lawsuits(federal orgs & projects)

Slides available at: