How should a buyer sample accessibility in assistant-prepared documents?

Headline statistic

Accessibility acceptance needs a declared document population, testable criteria, retained failures, and a named decision owner; a clean visual preview is not sufficient evidence.

Methodology: This protocol applies WCAG 2.2 concepts, W3C explanatory material, GAO data-reliability questions, and NIST governance controls to a bounded sample of assistant-prepared digital documents. It is an operating study, not a conformance certification or legal opinion. The unit is one final-format document linked to its approved source, template, conversion route, manual checks, assistive-technology observations, exceptions, reviewer, and disposition.

Key stats

• Population: every final-format document in one declared production lane, including corrected and rejected files.
• Evidence: source file, exported artifact, test record, issue log, repair evidence, and owner decision remain linked.
• Decision: accept, repair, replace, narrow the supported format, seek specialist review, or stop delivery.

Key takeaways

• Sample by document risk and feature, not convenience or appearance.
• Automated checks find some conditions but do not replace reading order, meaning, keyboard, or user-context review.
• The document owner sets the required standard and accepts residual risk; the assistant records and repairs within written authority.

Define the artifact before testing it

A buyer first needs to say what is being accepted. An editable word-processing file, a tagged PDF, a web page, and a slide deck expose different structures and controls. The production brief should name the final format, intended distribution channel, audience, required standard, supported viewing tools, and any approved alternative. If the assistant reviews the source file while recipients receive a converted PDF, the study has tested the wrong artifact. Keep the exported file and its source version together so a later correction can be reproduced.

The document inventory should describe features that can change the test: headings, tables, forms, links, images, charts, footnotes, columns, language changes, headers, and reading order. A five-page text memo is not equivalent to a two-page form. Sampling only the simplest files can create reassuring percentages while excluding the documents most likely to contain barriers. Stratification makes the claim modest and useful: the observed sample covered stated feature classes, not every possible reader, tool, or document.

Acceptance criteria must be written before results are known. WCAG provides testable web-content outcomes, while its Understanding material is informative rather than normative. A business may choose a relevant standard or specialist protocol for its document format. The assistant can map checks to that approved rule and preserve evidence. The owner or qualified reviewer decides whether the rule is appropriate, whether an alternate format is acceptable, and whether a consequential exception can be released.

Build a feature-stratified sample

Start with a complete release list for a fixed period. Tag each file by template, final format, length band, feature set, conversion method, novelty, and consequence if information is missed. Include every first use of a template and every known conversion failure, then draw ordinary examples from the remaining groups. This design deliberately overrepresents risky work, so its issue rate cannot be reported as the rate for all documents. It does, however, help a buyer decide where stronger gates or specialist review are warranted.

Retain failures, withdrawals, and repairs in the denominator. Removing a file after a problem is found turns the sample into a display of survivors. Record the original result, issue class, who decided the remedy, corrected artifact, and retest. If a deadline causes a document to be replaced with an accessible web page or plain-text version, that is an operational outcome rather than a passing result for the original file. The distinction protects both readers and the integrity of later improvement claims.

Reviewer agreement can be measured on a small overlapping subset. Two reviewers independently apply the same criteria, then discuss differences such as whether a heading communicates structure, whether link text has enough context, or whether a table linearises meaningfully. Disagreement is not automatically reviewer error. It may reveal an ambiguous rubric, a tool limitation, or a genuinely contextual judgment. Preserve the initial decisions and resolution instead of editing the record until reviewers appear to agree.

Test structure, meaning, and use

A visual inspection begins the work but cannot finish it. Check whether headings are encoded as headings, lists as lists, tables with the relationships needed for interpretation, and the document language is identified. Inspect reading order after conversion rather than assuming the source application preserved it. Images need alternatives that serve their actual purpose; decorative graphics should not create noise. A chart description must communicate the decision-relevant pattern without inventing precision that the chart itself does not support.

Interaction requires its own pass. Follow links from meaningful labels, move through controls in a logical order, confirm that focus is visible where the format supports interaction, and inspect fields for names, instructions, errors, and status changes. A mouse-only success does not show keyboard access. An exported form that looks correct but loses labels is a failed conversion. Record the tool, version, settings, and exact step so another reviewer can reproduce what happened rather than relying on a generic checked box.

Assistive-technology observation should answer a bounded question. For example: can a reviewer using the organisation’s supported screen reader identify the title, navigate headings, understand a sample table, reach each link, and complete a representative field? That observation is valuable but does not prove usability for every person, device, setting, or disability. Where the document is high consequence or the format is novel, include qualified accessibility review or user research instead of stretching an assistant sample into a certification claim.

Separate repair authority from release authority

The assistant can often repair known mechanical defects: apply an approved heading style, remove empty paragraphs used for spacing, replace an ambiguous link label, enter approved alternative text, or repeat an export using documented settings. Content meaning, reading sequence, mathematical notation, complex tables, legal notices, and material alternatives may require the author or specialist. The issue record should state which role owns the next decision. Repetition does not grant authority to resolve an ambiguous meaning.

A repair is complete only after the final artifact is retested. Correcting the source without regenerating the deliverable leaves conversion risk unexamined. Keep the failed artifact where retention rules permit, create the corrected version under a new version identifier, repeat the relevant checks, and document the result. If the fix alters wording or meaning, route it through editorial approval as well. Accessibility review is not permission to silently rewrite an approved statement.

Release status should communicate evidence, not optimism. Useful states include not tested, tested with stated scope, repair pending, specialist review required, alternate provided, accepted with an owner-approved exception, and rejected. Avoid a single green accessibility field that hides the standard, artifact, date, reviewer, and unresolved issue. The buyer should be able to trace exactly why a file was delivered and which questions remain open.

Interpret the result without overclaiming

Report coverage first: how many eligible files existed, how many were sampled in each feature class, which checks were applied, which tools were used, and what could not be tested. Then report findings by issue type and consequence, with repaired and unresolved results separated. A high pass share from plain memos says little about complex forms. A small number of serious reading-order failures may justify a process change even when most sampled checks passed.

The study cannot establish legal compliance, universal usability, or the accessibility skill of an individual worker. It can show whether a declared production lane preserves required evidence and catches specified defects under stated conditions. Tool updates, template changes, new conversion services, and changes in the document population all limit comparison over time. Re-run the sample when those conditions change and keep the earlier record rather than presenting a moving process as one continuous benchmark.

For a buyer, the practical outcome is a narrower and safer delegation decision. Routine structure checks and documented repairs may fit an assistant lane when examples and escalation rules are clear. Complex or consequential artifacts can be routed to qualified review. The owner retains responsibility for the standard, distribution decision, exceptions, and response when a recipient reports a barrier. That boundary makes assistant support useful without turning a checklist into unsupported assurance.

Apply the protocol to document production

Use this evidence model when scoping a document production service, while keeping standard selection, specialist interpretation, and final release with the named owner.

Related Research

Questions people ask