Email: Comments on WCAG 3.0 draft (3 - Testing)

[jspellman]

Comment from Email: from Word docx

2) Testing Two concerns arise around the use of the word testing in WCAG 3 draft.

1. Things seem to be grouped in testing that are not actually tests per se
2. Testing for conformance to a standard is conflated with testing to make sure a site is as accessible and usable as possible. These are two very different activities. Both important. But not that same. In some places Testing is used to describe something which is a real test. That is; a test to be sure that something does something? Does something meet a criterion? In other places (i.e. in the section on holistic tests) it says that testing includes ‘user-centered design methods’ and ‘usability testing’.
   • ‘User-centered design methods’ is not a test. Testing may be used as part of the user-centered design process, but user-centered design is not a test or a method for testing.
   • Usability testing is a means for testing, but not for conformance testing.
   o Usability testing is used for continuous quality improvement (CQI). These guidelines for example should be user tested to make sure that they are usable. However, they can still be an accurate standard of the highest quality, and not be usable by many people who just don't understand standards, or the technologies needed. You could also create guidelines that are very usable by people (for example by making them very short and simple) but have no validity or real use in testing whether something is accessible.
   Tests for conformance to a standard need to be restricted to tests that actually test conformance to the standard. I agree that ALL websites should be user tested - by people who have a full range of disabilities. But that is a completely different issue from testing the sites to see whether or not they conform to a standard. • Conformance testing determines if a website conforms to a standards. • User testing is done to improve the accessibility of the site and to identify issues that were overlooked or not covered by provisions in the standard – but still important. My concern is that the term testing is being redefined in different ways than normal for conformance testing - in order to try to achieve some goal that is outside the reach of an accessibility standard. For example, if one couldn't find any objectively testable provisions that would cover the needs of a particular disability group --- but one really want to include more provisions for that group…. one approach would be to re-define what is meant by testing to include different types of nonobjective testing. This would then allow one to include nonobjective provisions – which in turn allows more provisions for this group.” That is fine for a general guidance document – but not for a standard that one needs to conform to. I know the pain that can lead one to do this. We had the same problem in WCAG 2.0. We actually spent enormous time on cognitive language learning disabilities for example (more than on any other single disability) trying to find provisions that would address their needs and yet would be objective and meet the criteria necessary for a testable provision. We called in Nancy Ward, Clayton Lewis and a whole host of other people to talk with us and propose provisions that might work. John Slaton and I launched two, many-months-long efforts on both the cognitive language and learning disability area and the use of plain language in the guidelines. It was the most frustrating thing I have ever done in my life. Seeing the needs, but being unable to identify or find ways to qualify as strategies from all the materials we read, and people we talk to, was the most difficult and frustrating part of the work on WCAG.

[jspellman]

Thank you for your comment. Project members are working on your comment. You may see discussion in the comment thread and we may ask for additional information as we work on it. We will mark the official response when we are finished and close the issue.

[jspellman]

Personal comment (not as facilitator). An accessibility standard structure that excludes disabilities like cognitive disabilities is not acceptable to me personally. It has been affirmed as the consensus of the task force on multiple occasions. We have to do better. If the WCAG2 structure excludes the needs of people with cognitive disabilities, it is flawed and has to be addressed. We may not have it correct yet, but I think we are on the right track to include more disability needs without sacrificing the requirements of regulatory agencies. So far, the regulatory agencies that have responded have generally liked the direction of new WCAG3 conformance and want to see further development.