lauracarlson
commented
3 years ago Thank you for your comment. Project members are working on your comment. You may see discussion in the comment thread and we may ask for additional information as we work on it. We will mark the official response when we are finished and close the issue.
The conformance model and the related topic of scoring represents the most significant problems for WCAG 3.0. Flexibility for conformance is desperately needed. Full conformance is very difficult for almost every site currently. Related to the question posed, we support the idea of representative sampling, as an essential technique to assess the level of conformance for a site when testing every page isn’t possible (which is most of the time).
We are concerned that the small amount of Guidelines provided doesn’t provide enough context to really understand how the conformance model will work in practice. The rubric for rating the outcomes is a useful approach, but it assuming that methods are informative it will be necessary to ensure that the Outcome ratings provide sufficient information to allow an appropriate evaluation and measurement can be conducted.
We are also not clear on how are the thresholds for the adjectival ratings determined. Will the thresholds remain the same over time or will they adapt to technological changes?
The overall scoring under the conformance model needs to be more clearly based in evidence. The 3.5 value feels arbitrary. We suggest that the Working Group may need to conduct further research with functional groups to determine how well the user’s needs align with the outcomes that are linked to that group. For example, do we have evidence that issues with Text Alternatives impacts users in the “Cognitive – Mental Health” functional group to the same degree as those in “Sensory – Vision & Visual”? If not, why assess using the same rating scale or with the same final threshold value?
We are also concerned that the final number of outcomes and the process required to assess all of them may represent a substantially larger burden for conformance testing than WCAG 2.x – we fully support the inclusion of additional Outcomes to support additional functional groups, but this greater comprehensiveness will come at a cost to content providers so it is critical that the Outcomes are developed with thought toward easing the testing burden.
With this type of conformance, there is a possibility that the system may be gamed in order to make a conformance claim. For example, a site with many additional spacer images with appropriate null alt will more easily meet higher threshold values for the ratings. This may be unavoidable, but it might also identify outcomes that limit opportunities for rating improvements through gaming.