How we test
The exact checks behind every AccessFix report, how we rank what we find and what automated checks can’t tell you.
Which pages we check
The pages shoppers use on the way to buying: the home page, a collection, a product page and search results, plus the cart and checkout when the store owner asks for them (see below). Each page is checked twice: at desktop width (1280 pixels) and at phone width (390 pixels), because menus, drawers and buttons often change on small screens.
We visit public pages only, the way any shopper’s browser would. We never log in, submit forms or change anything.
Accessibility checks
Each page is tested with axe-core, the open-source rules engine used by most accessibility tools, set to the WCAG 2.0, 2.1 and 2.2 rules at levels A and AA. It finds problems such as images with no description, buttons and links with no name, low-contrast text, unlabelled form fields, missing page language and small tap targets.
At phone width we also check reflow (WCAG 1.4.10): at 320 pixels wide, nothing should force shoppers to scroll sideways.
Cart to checkout
With the store owner’s permission, we also walk the buying path: from a product page we add one item to a cart in our own browser session, check the cart and its checkout button, then open Shopify’s checkout. We never enter an email, address or payment details, and we empty the cart afterwards.
Shopify runs the checkout itself. Stores can change its branding but not its code, so problems we find there are reported to you to raise with Shopify, not fixed by us. Shopify’s robots.txt asks automated tools to stay out of carts and checkouts, which is why we only do this when you ask.
Testing by hand
Automated checks find only part of the problems. Every fix we make is also tested by a person:
- With a keyboard only: every menu, drawer, pop-up and button reachable and usable, with a visible focus outline
- With screen readers: VoiceOver on Mac and iPhone, and NVDA on Windows
- Zoomed to 200% and on a phone-sized screen
Security checks
The free scan is passive: it looks only at what any visitor’s browser already receives. We check:
- HTTPS, and that plain http:// visits are sent to https://
- JavaScript libraries such as jQuery and Bootstrap against published security bugs (CVEs)
- Secret keys left in page code or the store’s own scripts (shown masked, never in full)
- Files or forms loaded or sent without encryption
- Security headers: HSTS, clickjacking protection, file-type sniffing and a Content Security Policy
- Every third-party company whose code runs on the store
We never try to break in: no password guessing, no attack payloads, no hidden pages. Deeper reviews of apps, staff accounts and theme code happen only with written permission. Our scanner also refuses to visit private or internal network addresses.
How we rank problems
Accessibility problems are grouped by what they do to a shopper:
- Blocks shoppers: someone can’t finish buying, for example an unnamed cart button or a keyboard trap.
- Confusing: the store can be used, but with real difficulty, for example low-contrast prices.
- Polish: small issues worth fixing once the bigger ones are done.
Security findings are rated high, medium or low by what an attacker could do with them. Problems that appear in a shared header or footer are counted once, not once per page.
The WCAG 2.2 checklist
Every report lists all 55 WCAG 2.2 A and AA success criteria with one of four results:
- Fails: an automated check failed on at least one page.
- Needs review: the automated check couldn’t decide; a person has to look.
- No automated failures: the automated checks for it passed. This is not the same as meeting the criterion.
- Needs a person: no automated check exists for it, for example keyboard use or focus order.
It shows progress. It is not a compliance statement.
Proving the fixes
After fixing, we scan the same pages again and compare the two scans element by element. You get a before-and-after report showing what was fixed and anything new, and the fixes are re-tested by hand before you publish the theme.
What this can’t tell you
- Automated tools find only part of accessibility problems. A clean scan does not mean a store is fully accessible.
- A passive security check can’t prove a store is safe. A clean result doesn’t mean a site can’t be hacked.
- Reports are not legal advice and don’t certify compliance with the ADA, the European Accessibility Act or WCAG.
Want to see the result? Open a sample report or get a free scan.