Why Accessibility Testing Is No Longer Optional
Web accessibility lawsuits hit record numbers in 2024, with over 4,600 federal cases filed against companies whose websites failed to meet basic accessibility standards. The European Accessibility Act enters full enforcement in 2025. The US Department of Justice updated its guidance to explicitly include web content under the Americans with Disabilities Act. If you build websites and you are not testing for accessibility, you are shipping legal liability along with every deployment.
But compliance pressure is only part of the story. Roughly 16% of the global population lives with some form of disability. Screen readers, keyboard navigation, voice control, and switch devices are not edge cases — they are everyday tools for millions of people. An inaccessible form, a missing alt attribute, or a broken focus order does not just fail an audit. It locks real people out of your product.
The problem is that manual accessibility testing is slow, inconsistent, and expensive. A trained auditor testing a single page against WCAG 2.1 Level AA can spend 30 to 60 minutes working through checkpoints. Multiply that by hundreds of pages across dozens of deployments per month, and manual testing collapses under its own weight. Automated accessibility testing does not replace human judgment entirely, but it catches 30 to 57 percent of WCAG violations instantly — and it catches them on every commit, every pull request, and every deployment without anyone needing to remember to check.
This guide covers the practical tooling that makes automated accessibility testing work: axe-core for rule-based analysis, Lighthouse for performance-aware auditing, and CI pipeline integration that prevents accessibility regressions from reaching production. If you are already building CI/CD pipelines with GitHub Actions, adding accessibility gates is straightforward. If you are not, this is a strong reason to start.
Understanding WCAG and What Automation Can Actually Catch
The Web Content Accessibility Guidelines (WCAG) define three conformance levels: A, AA, and AAA. Most legal requirements and industry standards target Level AA, which includes 50 success criteria covering perceivable, operable, understandable, and robust content. Not all of these criteria can be evaluated by machines.
Automated tools excel at detecting structural issues: missing alternative text on images, insufficient color contrast ratios, empty links and buttons, missing form labels, duplicate element IDs, incorrect ARIA attribute usage, and broken heading hierarchies. These are the violations that appear in code and can be checked against deterministic rules. A machine can calculate that a 2.8:1 contrast ratio fails the 4.5:1 minimum for normal text. It cannot determine whether an image’s alt text actually describes the image meaningfully.
This distinction matters because it sets realistic expectations. Automation handles the mechanical checks — the low-hanging fruit that accounts for the majority of accessibility failures on the average website. The remaining violations require manual testing: logical reading order, keyboard trap identification in complex widgets, meaningful sequence of content, and whether alternative text conveys the right information. A strong accessibility strategy uses automation to handle the volume and manual audits to handle the nuance.
If you are working with modern component-based architectures, testing accessibility at the component level through tools like Storybook catches issues before they propagate across your application. Combined with Web Components and Shadow DOM, encapsulated components need particular attention to ensure ARIA attributes pierce shadow boundaries correctly.
axe-core: The Industry Standard Accessibility Engine
axe-core, maintained by Deque Systems, is the most widely adopted open-source accessibility testing engine. It powers the accessibility auditing in Chrome DevTools, Firefox DevTools, Lighthouse, and dozens of third-party testing tools. The engine evaluates rendered DOM against a comprehensive rule set mapped to WCAG 2.0, 2.1, and 2.2 success criteria, Section 508 requirements, and best practices.
What makes axe-core effective is its approach to results. Rather than flooding you with false positives, axe-core categorizes findings into violations (definite failures), passes (definite successes), incomplete (needs manual review), and inapplicable (rules that do not apply to the page). This categorization reduces alert fatigue and builds trust in the tool — when axe-core reports a violation, it is almost always a real problem.
The engine runs in any JavaScript environment. You can use it in the browser through the axe DevTools extension, in Node.js through programmatic APIs, or within testing frameworks like Playwright, Cypress, Puppeteer, and WebDriverIO. This flexibility means axe-core fits into whatever testing infrastructure you already have.
Integrating axe-core With Playwright for End-to-End Accessibility Testing
If you are already running end-to-end tests with Playwright, adding accessibility checks requires minimal additional setup. The @axe-core/playwright package wraps the axe-core engine and provides a clean API for running accessibility audits within your existing test suites.
// accessibility.spec.ts
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
// Test critical pages for WCAG 2.1 Level AA compliance
const criticalPages = [
{ name: 'Homepage', path: '/' },
{ name: 'Sign Up', path: '/signup' },
{ name: 'Dashboard', path: '/dashboard' },
{ name: 'Settings', path: '/account/settings' },
{ name: 'Checkout', path: '/checkout' },
];
for (const page of criticalPages) {
test(`${page.name} should have no critical accessibility violations`, async ({ page: browserPage }) => {
await browserPage.goto(page.path);
// Wait for dynamic content to finish rendering
await browserPage.waitForLoadState('networkidle');
const results = await new AxeBuilder({ page: browserPage })
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
.exclude('.third-party-widget') // Exclude elements you cannot control
.analyze();
// Log detailed violation info for debugging
if (results.violations.length > 0) {
const summary = results.violations.map(v => ({
rule: v.id,
impact: v.impact,
description: v.description,
nodes: v.nodes.length,
help: v.helpUrl,
}));
console.table(summary);
}
// Fail only on serious and critical violations
const seriousViolations = results.violations.filter(
v => v.impact === 'critical' || v.impact === 'serious'
);
expect(seriousViolations).toHaveLength(0);
});
}
// Test specific interactive components after user actions
test('modal dialog should maintain focus trap and be accessible', async ({ page }) => {
await page.goto('/dashboard');
await page.click('[data-testid="open-modal"]');
// Audit only the modal component
const results = await new AxeBuilder({ page })
.include('.modal-overlay')
.withTags(['wcag2a', 'wcag2aa'])
.analyze();
expect(results.violations).toHaveLength(0);
});This test suite covers several important patterns. It iterates through critical pages rather than testing a single route, which ensures broad coverage without writing repetitive test cases. The withTags method filters rules to WCAG 2.1 Level AA, which is the most common compliance target. The exclude method lets you skip third-party widgets and embedded content that you cannot fix. Filtering for serious and critical impacts prevents the build from breaking on minor issues while still catching the violations that affect real users.
The modal test demonstrates component-level testing after user interaction. Many accessibility failures only appear after state changes — opening modals, expanding accordions, toggling menus. Testing the initial page load is not enough. You need to audit the DOM in its various interactive states.
Lighthouse Accessibility Auditing at Scale
Google Lighthouse includes an accessibility category that runs a subset of axe-core rules alongside its own checks. While axe-core gives you the deepest accessibility analysis, Lighthouse adds value by combining accessibility scores with performance, SEO, and best practice metrics in a single report. This holistic view helps teams understand how accessibility fits within broader site quality — a topic we explore further in our guide to SEO for JavaScript SPAs, where accessibility and discoverability intersect directly.
Lighthouse runs in Chrome DevTools, as a Node.js CLI tool, or through the PageSpeed Insights API. For automation purposes, the CLI and Node.js module are most useful. They produce JSON reports that can be parsed, stored, and compared across builds.
The Lighthouse accessibility score is calculated by weighting individual audit results by their impact. A page with one critical violation and ten passing audits might score 72, while a page with three minor issues could score 95. The scoring is not linear, and it is not a direct representation of WCAG conformance. A Lighthouse score of 100 does not mean your page is fully accessible — it means the automated checks passed. Use the score as a trend indicator, not a compliance certificate.
For teams that need to audit large numbers of pages, Lighthouse CI provides a structured way to run audits in continuous integration, store historical results, and set performance and accessibility budgets that fail builds when thresholds are not met. This is particularly important for content-heavy sites where new pages, blog posts, or product listings might introduce accessibility regressions that affect responsive layouts across different viewport sizes.
CI Integration: Making Accessibility Gates Mandatory
The highest-value step in accessibility testing automation is integrating it into your CI/CD pipeline as a required check. When accessibility tests run on every pull request and block merging on failure, accessibility stops being an afterthought and becomes a first-class quality gate — the same way unit tests or linting prevent code regressions. If you are already using ESLint and Prettier for code quality, adding accessibility checks follows the same pattern of automated enforcement.
The goal is a pipeline that runs accessibility audits automatically, reports results clearly, and prevents regressions from reaching production. Here is a practical GitHub Actions workflow that combines axe-core testing through Playwright with Lighthouse CI scoring.
# .github/workflows/accessibility.yml
name: Accessibility Testing
on:
pull_request:
branches: [main, develop]
push:
branches: [main]
jobs:
accessibility-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build application
run: npm run build
- name: Start preview server
run: npm run preview &
env:
PORT: 4173
- name: Wait for server
run: npx wait-on http://localhost:4173 --timeout 30000
# Run Playwright accessibility tests with axe-core
- name: Run axe-core accessibility tests
run: npx playwright test --project=accessibility
continue-on-error: false
- name: Upload axe-core test results
if: always()
uses: actions/upload-artifact@v4
with:
name: accessibility-results
path: test-results/
# Run Lighthouse CI for accessibility scoring
- name: Run Lighthouse CI
uses: treosh/lighthouse-ci-action@v11
with:
urls: |
http://localhost:4173/
http://localhost:4173/signup
http://localhost:4173/dashboard
configPath: .lighthouserc.json
uploadArtifacts: true
# Post results as PR comment
- name: Comment accessibility results on PR
if: github.event_name == 'pull_request'
uses: marocchino/sticky-pull-request-comment@v2
with:
header: accessibility
message: |
## Accessibility Test Results
**axe-core:** ${{ steps.axe.outcome == 'success' && 'Passed' || 'Failed' }}
**Lighthouse Accessibility Score:** See artifacts for details
View full results in the [Actions tab](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})This workflow runs on every pull request and push to main. The axe-core tests through Playwright catch specific WCAG violations with detailed failure reports. Lighthouse CI provides trend-tracking accessibility scores. The combination gives both precision (axe-core) and breadth (Lighthouse) in a single pipeline.
The Lighthouse CI configuration file controls budgets and assertions. A minimal but effective configuration sets a floor for the accessibility score and warns when it drops.
Setting Accessibility Budgets With Lighthouse CI
Create a .lighthouserc.json file in your project root to define accessibility score thresholds. This configuration tells Lighthouse CI to fail the build if the accessibility score drops below 90 and warn if it drops below 95.
The budget approach works well for teams adopting accessibility incrementally. Start with a threshold that matches your current score, then ratchet it upward as you fix violations. A team scoring 78 today should set the failure threshold at 75 — preventing regressions — while working toward 90. Attempting to enforce 100 from day one creates frustration and leads teams to disable the check entirely.
Beyond Automated Testing: Building an Accessibility Culture
Automated testing catches the mechanical failures, but building truly accessible products requires a cultural shift. Developers need to understand why accessibility matters, not just how to pass the tests. Here are practices that reinforce accessibility beyond the CI pipeline.
Developer Education and Awareness
Run screen reader testing sessions where developers navigate their own features using VoiceOver, NVDA, or JAWS. Nothing motivates accessibility fixes faster than personally experiencing a broken tab order or an unlabeled button. Even 30 minutes with a screen reader transforms how developers think about semantic HTML.
Include accessibility acceptance criteria in user stories. Instead of a vague requirement like “make it accessible,” write specific criteria: “form fields have visible labels associated via the for attribute,” “error messages are announced to screen readers via aria-live regions,” “all interactive elements are reachable via keyboard tab navigation.” Managing these acceptance criteria alongside development tasks is where project management tools prove their value — platforms like Taskee let teams track accessibility requirements as concrete, checkable items within their development workflow rather than treating them as an afterthought.
Component-Level Accessibility Testing
Testing accessibility at the component level prevents violations from propagating. When a button component lacks proper ARIA attributes, every instance of that button inherits the problem. Fix it once at the component level, and every page benefits. Storybook’s accessibility addon runs axe-core against individual component stories, flagging violations during development rather than after deployment.
Design systems should encode accessibility requirements directly. A form field component should require a label prop and throw a development-mode warning if one is not provided. A modal component should manage focus trapping internally. An icon button should require an aria-label. When accessibility is built into the component API, developers cannot accidentally skip it.
Manual Testing Cadence
Automated tests are continuous. Manual audits should be periodic. A quarterly accessibility review of key user flows — registration, checkout, primary feature paths — catches the issues automation misses. Tab through your entire checkout flow using only a keyboard. Navigate your dashboard with a screen reader. Resize text to 200% and verify that content remains readable without horizontal scrolling.
Consider integrating manual testing checkpoints into your monitoring and observability practices. Accessibility is a dimension of application health that deserves tracking alongside uptime, performance, and error rates.
Common Accessibility Violations and How to Fix Them
Automated tools report the same categories of violations repeatedly across projects. Knowing the most common failures helps you write accessible code from the start rather than fixing it after the audit.
Missing or Empty Alternative Text
Images without alt attributes or with empty alt text on meaningful images are the single most common violation. Decorative images should have alt="" (empty, not missing). Informative images need descriptive text. Complex images like charts need longer descriptions via aria-describedby or a linked text alternative. The axe-core rule image-alt catches missing attributes but cannot evaluate whether existing alt text is accurate — that requires human review.
Insufficient Color Contrast
WCAG 2.1 requires a minimum contrast ratio of 4.5:1 for normal text and 3:1 for large text (18px bold or 24px regular). Gray text on white backgrounds is the most common offender. Light placeholder text in form fields frequently fails. Automated tools calculate contrast ratios precisely, making this one of the easiest violations to detect and fix. Use tools like the WebAIM Contrast Checker during design to prevent contrast issues from reaching code.
Missing Form Labels
Form inputs without associated labels are invisible to screen readers. The placeholder attribute is not a substitute for a label — placeholders disappear when the user begins typing, and screen readers do not consistently announce them. Every input, select, and textarea needs a <label> element with a matching for attribute, or an aria-label for inputs where a visual label is genuinely inappropriate (search fields, for example).
Empty Links and Buttons
Links and buttons that contain only an icon without text are announced as “link” or “button” by screen readers, giving users no indication of what the element does. Icon buttons need aria-label attributes. Links wrapping images need alt text on the image or an aria-label on the link. The axe-core rules link-name and button-name detect these reliably.
Broken Heading Hierarchy
Jumping from h2 to h4, using headings purely for visual styling, or having multiple h1 elements creates navigation problems for screen reader users who rely on heading structure to understand page organization. Headings should form a logical outline: one h1 per page, h2 for major sections, h3 for subsections within those sections. The axe-core rule heading-order flags hierarchy violations.
Measuring Progress and Setting Targets
Accessibility improvement is a continuous process, not a one-time fix. Teams that succeed track metrics over time and set incremental targets rather than attempting to fix everything at once.
Start by baselining your current state. Run axe-core across your critical pages and record the total number of violations by impact level. Track this number weekly. A dashboard showing violation count over time makes progress visible and motivates continued investment.
Set realistic thresholds. A site with 200 violations should not target zero violations in the next sprint. Aim for a 25 percent reduction per quarter. Prioritize critical and serious violations first — these have the highest user impact. Minor and moderate violations can be addressed as part of regular code maintenance.
Lighthouse CI’s historical tracking is valuable here. When you run Lighthouse CI on every deployment and store results, you can see exactly when accessibility scores changed, which commits caused regressions, and whether the overall trend is positive. This data transforms accessibility from a subjective concern into an objective, measurable quality metric.
For teams building web applications with modern JavaScript frameworks, accessibility testing is especially important. Single-page applications that manage routing, focus, and dynamic content client-side introduce accessibility challenges that traditional server-rendered pages avoid. Testing frameworks need to account for these patterns, verifying that route changes announce new content, that dynamically rendered elements receive focus appropriately, and that loading states communicate progress to assistive technology users.
Frequently Asked Questions
What percentage of WCAG violations can automated accessibility testing catch?
Automated accessibility testing tools like axe-core and Lighthouse can detect approximately 30 to 57 percent of WCAG 2.1 Level AA violations, depending on the complexity of the page and the specific rules implemented. These tools excel at catching structural issues such as missing alt text, insufficient color contrast, empty form labels, broken heading hierarchies, and incorrect ARIA attribute usage. The remaining violations require manual testing, including evaluating meaningful content order, keyboard trap identification in complex widgets, and whether alternative text accurately describes visual content.
How do axe-core and Lighthouse differ for accessibility testing?
axe-core is a dedicated accessibility testing engine that provides deep, comprehensive analysis with detailed violation reports categorized by impact level. Lighthouse uses a subset of axe-core rules as part of a broader site quality audit that also includes performance, SEO, and best practices. Use axe-core when you need detailed, granular accessibility testing integrated into your test suites. Use Lighthouse when you want a high-level accessibility score alongside other quality metrics. For maximum coverage, use both: axe-core through Playwright or Cypress for specific WCAG compliance testing, and Lighthouse CI for trend tracking and accessibility budgets.
How do I integrate accessibility testing into a CI/CD pipeline?
Add accessibility testing as a required check in your CI pipeline alongside unit tests and linting. Install @axe-core/playwright or cypress-axe to run axe-core within your existing end-to-end tests. Use Lighthouse CI via the treosh/lighthouse-ci-action GitHub Action for score tracking and budget enforcement. Configure your pipeline to fail on critical and serious violations while allowing minor issues to pass with warnings. Start with a threshold that matches your current accessibility score, then increase it incrementally as your team fixes violations.
Should I test accessibility at the component level or the page level?
Both levels are necessary for thorough coverage. Component-level testing using tools like Storybook’s accessibility addon or jest-axe catches violations at their source, preventing a broken component from propagating issues across every page that uses it. Page-level testing with Playwright or Cypress validates that components work correctly together, that page structure is sound, and that interactive states such as modals, dropdowns, and dynamic content updates remain accessible. Component testing catches issues early during development, while page testing catches integration-level problems and validates the complete user experience.
What is a realistic accessibility score target for an existing application?
Start by baselining your current Lighthouse accessibility score and total axe-core violation count. Set your initial CI threshold slightly below your current score to prevent regressions, then increase it by 5 to 10 points per quarter. For most teams, a Lighthouse accessibility score of 90 or above is an achievable intermediate target. Reaching 100 requires addressing every automated check, which is feasible for new projects but can take six to twelve months for established applications. Remember that a perfect automated score does not guarantee full WCAG compliance — manual testing is still required for the 40 to 70 percent of criteria that automation cannot evaluate.
