Assess the holistic cost of transition

[rachaelbradley]

Discuss the holistic cost of the new conformance model. Do we think the proposed model will increase the level of effort of accessibility testing such that it would be a barrrier to adoption? See meeting minutes https://www.w3.org/2021/11/23-ag-minutes.html#item04 and #369

[yatil]

Having worked as a solo accessibility consultant and as part of small accessibility companies, the amount of tracking and accounting for conformance is probably unsustainable for most of the situations I have worked with. Counting the number of conforming vs. nonconforming images, for example, puts a burden on us. There is little use for a client to know that 80% or 90% of the images conform to the standard.

I also suspect that we would need to have proof of the individual instances of an issue, which means archiving versions of the site. Many clients don’t even provide us with finished products, sometimes code changes during an audit.

While I can see that such a conformance model can work for big companies that work mostly with automated testing, and have the means to build or acquire such testing facilities, for smaller companies and auditors, WCAG 3 feels more accounting. I would propose to spin out evaluation and conformance from W3CAG and provide conformance methodologies that are simplified and detailed as separate specifications.

[bruce-usab]

Erik, FWIW, my understanding is that it is also possible for the W3CAG conformance model to be compatible with maturity metrics such as many U.S. federal agencies are doing. Currently, there is reporting twice a year on five metrics using a five point Likert-style scale. See at Section508.gov: Assessing Program Maturity

[hidde]

I would like to echo @yatil's point that WCAG 3 feels like more accounting. I feel it does, in many aspects, and that this puts it over the point where my clients would:

• have their sites audited less often
• choose not to have their site audited at all
• pick a supplier that cuts costs by delivering lower quality reports

I worry that this may result in less accessibility overall, rather than more and encourage the Working Group to come up with a much simplified conformance model, like the one that exists for WCAG 2.

[yatil]

@bruce-usab If I understood the W3CAG draft correctly (and I might not) the “Program Maturity” would fall under holistic tests that will be explored further and are needed to meet Silver or Gold levels; for Bronze, you’d need still the WCAG accounting. I think only a fraction of clients would agree to such “holistic tests” and I guess for smaller clients Silver and Gold levels are pretty much unobtainable.

I agree with @hidde. Many practitioners might also recommend sticking to WCAG 2 conformance because it is simpler (and less costly) to test, maybe with some W3CAG enhancements.

And this all ignores the additional cost for practitioners and accessibility consultancies to test two sets of rules, write reports in two completely different ways, keep maybe different systems for different versions of W(3)CAG.

[bruce-usab]

@yatil — as you describe, yes, that comports with my understanding of the expectations for Silver and Gold. Much is still to be sorted out, but your reply mentioned holistic tests needed to meet Silver or Gold, and I believe we are thinking along the same lines. Your earlier observation that counting the number of conforming vs. nonconforming images as burdensome, while an absolutely legitimate concern, IMHO is tangential to this particular issue thread. OTOH, I might be reading too much into the title of this particular issue thread…