Define conformance classes

[annevk]

Per IRC discussion with @jyasskin @domenic @sideshowbarker it seems good to flush out conventions around conformance classes (see https://html.spec.whatwg.org/multipage/infrastructure.html#conformance-classes and https://www.w3.org/TR/qaframe-spec/#specifying-conformance for an introduction to the topic).

Questions that we should seek to resolve:

• Does it make sense to require things from web developers (it was suggested to require things from documents instead, but that doesn't quite work for APIs)?
• Does it make sense to require things from other standards/specifications? (This is somewhat rhetorical since we need this to ensure new formats use UTF-8 and such; the main question is how we go about it.)

cc @karlcow

[sideshowbarker]

We should note that the IRC discussion started at http://logs.glob.uno/?c=freenode%23whatwg&s=29+Sep+2017&e=1+Oct+2017#c1038858

Would also be good to feedback from @marcoscaceres on this

[marcoscaceres]

Does it make sense to require things from web developers (it was suggested to require things from documents instead, but that doesn't quite work for APIs)?

I'm personally not a fan of making authors into conformance classes: unlike with user agents, such conformance requirements are unenforceable and untestable.

We (standards Editors) might have ideas about best practice, but developers often find noble ways of using APIs, markup, etc. leading to either willful violations in API usage, or semantic drift in case of markup - both which I see as good things and natural evolutions of technology.

Having said that: if you want to have conformance requirements for a "conformance checker" or "validator", then I think that's a totally valid class of product.

Does it make sense to require things from other standards/specifications? (This is somewhat rhetorical since we need this to ensure new formats use UTF-8 and such; the main question is how we go about it.)

Yes... how we go about it depends on the underlying standard, I guess: for instance, in WebIDL, there are rules that are enforceable in tooling (e.g., don't put [NoInterfaceObject] on a Dictionary).

I know the above is very "the tools will save us"... but in our case, it's kinda "the tools + the proactive community... maybe" will save us. That is, we need to, as a community, get really behind supporting the tools that can help us enforce core principles (e.g., webidl2.js and @plinss IDL parser) - and then maybe build on top of those, as we've been doing with Bikeshed and ReSpec to yell at spec editors when they do the wrong thing.

Obviously, we won't be able to computationally yell about everything... but it helps enforce some things.

[karlcow]

Does it make sense to require things from web developers (it was suggested to require things from documents instead, but that doesn't quite work for APIs)?

Yes. The QA framework is in general silent of what is valid to require as conforming, but is clear that if you expect a conformance of any kind, you need to be explicit and non-ambiguous about it. Indeed for WAI, all of these are requirements on another products which is in the end a document, but request certain things from Web developers in the case of WAI.

The details of types of products that can be conforming are more detailed in Variability in Specifications.

To reply to @marcoscaceres

not a fan of making authors into conformance classes

I have difficult to imagine how would this even work. It doesn't really make sense. Web authors MUST wear a blue beany on April 1st. On the other hand, it is totally appropriate to say that whatever tool the author is using (be is fingers with his brain or a IDE or something else) that the code/markup produced must conform to this rule X.

Does it make sense to require things from other standards/specifications? (This is somewhat rhetorical since we need this to ensure new formats use UTF-8 and such; the main question is how we go about it.)

Yes. XML InfoSet, QA framework are examples of that. How you go about it. The spec would say that you need to conform to XML InfoSet. We do that all the time, calling specs by reference.

All of that said I probably need to better understand the issue at stake. If someone can give me more details, I'm happy to go over it.

Thanks for the ping @annevk

[annevk]

@karlcow the idea behind the Infra Standard is to centralize language common to lots of standards. So the idea came up to define a set of conformance classes here that then can be reused by reference in other standards, such as HTML and Compat. That also gives a more coherent view on what conformance classes are acceptable.

[karlcow]

@annevk I don't see any issues into creating this. Uniformity of vocab and prose usually helps with understanding. I would recommend to read in details Variability in Specifications.