quailjs / quail

Accessibility testing in the browser and on the server.
http://quailjs.org
Other
300 stars 44 forks source link

WCAG2: add tests on an higher level: by success criterion #129

Open hannolans opened 10 years ago

hannolans commented 10 years ago

WCAG2 defines several techniques to comply to a success criterion. Most of the time one of the techniques is enough, sometimes a combination or two techniques, or depending on a situation. We could integrate alle the tests in one big test, or we can add a test on SC level that delegates portions to other tests. The last option is conform EARL (TestRequierement and TestCase). It gives more flexibility.

earl:TestRequirement
    Test Requirement - a higher-level requirement that is tested by executing one or more sub-tests. 
For example WCAG 2.0 Success Criterion 1.1.1, which is evaluated using several Techniques for Success Criterion 1.1.1 and combining the results.
earl:TestCase
    Test Case - an atomic test, usually one that is a partial test for a requirement. 
For example, Technique H36: Using alt attributes on images used as submit buttons provides a partial test for WCAG 2.0 Success Criterion 1.1.1. 

http://www.w3.org/TR/EARL10-Schema/#TestCriterion

hannolans commented 10 years ago

This could be a test loop for SC 1.2.2 captions (http://www.w3.org/TR/UNDERSTANDING-WCAG20/media-equiv-captions.html)

  1. test whether the page contains video content yes / no -> not applicable
  2. test whetther a closed caption file is provided no / yes -> 5
  3. test whether the video is an alternative for the text and labeled as such no / yes -> pass
  4. test whether the video contains open captions yes /no -> 5
  5. test whether all speech has captions yes/no -> fail
  6. test whether the caption text is identical to the spoken text yes -> pass /no -> fail
jessebeach commented 10 years ago

This depends on #101.

Can we introduce a group tag? And then each group needs logic e.g. if 1 of 3 passes, then the Success Criterion is met; or all 3 must pass for the Success Criterion to pass.

jessebeach commented 10 years ago

And then what happens if the testability is less than one.

kevee commented 10 years ago

Instead of just an array of techniques in our guideline definitions, we could instead have:

x.x.x:
  techniques:
    and:
      -T1
      -T2
      -T3
    or:
      -T4
      -T5

With nested and/or groups it could get a little complicated though. Perhaps it would be best for us to first do the work of identifying all the logic in techniques first, the above approach would work fine in there is no need for nested logic.

jessebeach commented 10 years ago

I'll introduce the SuccessCriterion object against this issue.

jessebeach commented 10 years ago

Working in branch issue-129

jessebeach commented 10 years ago

The evolving Pull Request:

https://github.com/quailjs/quail/compare/issue-129?expand=1