act-rules / act-rules.github.io

Accessibility conformance testing rules for HTML
https://act-rules.github.io/
Other
136 stars 68 forks source link

Boundary / Verbose examples #2185

Open Jym77 opened 5 months ago

Jym77 commented 5 months ago

Playing around the idea of boundary examples as discussed in #2164.

This adds one "boundary example" to some of the rules, showing how the rules are partial check and stuff that is inapplicable or even passing might still fail the SC (or other requirements).

These "boundary examples" must not be part of the normal examples that are required for tools to claim "consistent implementation" of the rule. Thus, tools can be more clever than the rule (i.e. fail these cases), but still have a consistent implementation.

This PR only shows a possible way to show these examples: in a collapsible section after the normal examples (so they are still together with the rule itself, but do not take space for people not interested in them). If we decide to go that way, we would need to build the website code (and tweak the validation) to automate part of this process.

I do believe that this won't require a format update since the format doesn't prohibit "extra" examples. Another possibility would be to place the boundary examples in the Background, but that makes them show a bit earlier in the text of the rule (even if the section is collapsed).

Closes issue(s):

Need for Call for Review: This will require a 2 weeks Call for Review (new concept)


Pull Request Etiquette

When creating PR:

After creating PR:

When merging a PR:

How to Review And Approve

giacomo-petri commented 3 months ago

I believe there is no need to clarify the outcome of each specific example for the specific rule (passed, failed, inapplicable) within a heading.

By definition, boundary examples "are not normative and are not required for consistent implementations." Therefore, in my opinion, and as recommended by @dan-tripp-siteimprove, even though I understand the intention to emphasize whether the current scenario is passing or inapplicable, we should avoid distinguishing this in the headings hierarchy to prevent confusion.

The reason why an example is an edge case, in my opinion, should be provided in the description of the example, leaving just "Boundary Example 1," "Boundary Example 2," etc., as titles.

giacomo-petri commented 3 months ago

And sorry, I forgot to mention the most important thing: I really love this idea! We should definitely move forward with this idea!

Jym77 commented 3 months ago

Just leaving my 2 cents.

  1. I don't think we should put details / summary in here. I'd rather have that be added by our build tool. It's less error prone that way, and means we can more easily make changes to it in the future.
  2. I think we should call these "Miscellaneous examples". As I understand the conversation, we'll want to use this for a variety of different kinds of examples. The only thing that they have in common is that they're not passed / failed / inapplicable examples.
  3. I feel we should use different H3s / H4s here. Calling these a passed example, or an inapplicable example doesn't set it apart from the rest of the rule sufficiently in my opinion. People will get confused by what these passed examples mean compared to the other passed examples. We should call them something else. There are lots of things we can use here, like "Out of scope example 1", "Accessibility support example 2", "Explicit semantic role example 3", "Shadow DOM example 4". etc.
  1. Definitely. This was just to show how it can look like.
  2. 👍
  3. I think it can be good for tools vendors to have a way to retrieve these cases and use them for validation purpose. This will require us to provide the rule's result in a way that the build tool can understand and forward. But I agree that the names can be confusing.