Automation Guides
Pragmatic, developer‑friendly automation patterns that catch low‑hanging issues early—without pretending automation equals WCAG compliance.
What automation is (and isn’t)
Automation is great for fast feedback, preventing obvious regressions, and nudging better habits in CI. It does not replace manual checks, assistive tech testing, or design reviews.
- ✅ Catches missing labels, low‑contrast regressions, ARIA misuse patterns, obvious keyboard traps.
- ❌ Does not determine meaningful name, role, value for complex widgets, logical focus order, or task success.
Quick start: add an a11y check to your UI tests
Python · Playwright + axe-core
Runs a basic axe scan on a page (or after an interaction). Fails the test on violations.
# requirements: playwright, pytest, axe-core-js via CDN or local bundle
# Example uses Playwright Python and injects axe.min.js
from playwright.sync_api import sync_playwright
import json, sys
AXE_CDN = "https://cdnjs.cloudflare.com/ajax/libs/axe-core/4.9.1/axe.min.js"
def run_axe(page):
page.add_script_tag(url=AXE_CDN)
# Configure rules if needed, e.g., to disable color-contrast while stabilizing CI
result = page.evaluate("""async () => {
return await axe.run(document, { resultTypes: ['violations'] })
}""")
return result
def main():
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("https://example.com")
result = run_axe(page)
violations = result.get("violations", [])
if violations:
print(json.dumps(violations, indent=2))
browser.close()
sys.exit(1)
browser.close()
if __name__ == "__main__":
main()
Tip: Start with p.wait_for_selector before scanning to avoid false positives on loading states.
C# · Playwright + axe-core
Same idea in .NET—inject axe, run, and fail on violations.
using Microsoft.Playwright;
using System.Text.Json;
using System.Collections.Generic;
using System;
using System.Threading.Tasks;
public class AxeResult { public List<object> violations { get; set; } = new(); }
public static class Program
{
public static async Task Main()
{
using var pw = await Playwright.CreateAsync();
await using var browser = await pw.Chromium.LaunchAsync(new() { Headless = true });
var page = await browser.NewPageAsync();
await page.GotoAsync("https://example.com");
var axeCdn = "https://cdnjs.cloudflare.com/ajax/libs/axe-core/4.9.1/axe.min.js";
await page.AddScriptTagAsync(new() { Url = axeCdn });
var json = await page.EvaluateAsync<string>(@"async () => {
const res = await axe.run(document, { resultTypes: ['violations'] });
return JSON.stringify(res);
}");
var result = JsonSerializer.Deserialize<AxeResult>(json)!;
if (result.violations.Count > 0)
{
Console.WriteLine(json);
Environment.Exit(1);
}
}
}
Alternative: If your org prefers Selenium, inject axe using IJavaScriptExecutor and run similarly.
CI pipeline examples
GitHub Actions
name: a11y-checks
on: [push, pull_request]
jobs:
axe:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: '3.12' }
- run: |
pip install playwright
python -m playwright install --with-deps chromium
- run: python scripts/axe_check.py
Keep it fast: run on PRs, scope to changed apps, and fail hard on critical violations.
Azure DevOps
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- task: UsePythonVersion@0
inputs:
versionSpec: '3.12'
- script: |
pip install playwright
python -m playwright install --with-deps chromium
python scripts/axe_check.py
displayName: 'Run axe check'
Start with a single “smoke route” per app. Expand coverage after you stabilise noise.
Suggested maturity path
- Lint & static checks (eslint-plugin-jsx-a11y, stylelint‑a11y) to block obvious anti‑patterns pre-commit.
- Page‑level axe scans in CI on a handful of canonical flows (home, product, checkout, login).
- Component‑level tests (Storybook + axe add‑on, unit tests asserting ARIA props).
- Visual contrast gating (design tokens + automated contrast on token changes).
- Hybrid flows: combine automation + manual AT spot checks before release.
Set smart thresholds
Use “quality gates” that reflect risk—not vanity metrics.
- Block PRs on critical rules (e.g., interactive controls without names).
- Warn on moderate rules initially; convert to blocking after remediation trend improves.
- Track
violations per routerather than a single global count.
FAQs
Do automated checks make us WCAG compliant?
No. They find a subset of issues. Compliance needs manual + AT testing.
Can we start small?
Yes—start with 1–2 routes and 1 browser in CI, prove value, then scale.
Which tools?
axe‑core for rules; Playwright for navigation and state; optional Storybook for components.