Create validation section

[mlagally]

Describe how validation has to be performed, several levels, tooling?

[mlagally]

Taking in a comment from @benfrancis in https://github.com/w3c/wot-profile/issues/10, for further discussion here:

Another question which is yet to be answered is what a WoT Consumer should do if a Thing Description fails to validate against one of the constraints in the proposed data model. E.g. if one affordance is missing a title or description member, does it render the entire Thing Description invalid and the Consumer should stop processing it altogether and return an error?

[mlagally]

We have two basic alternatives:

1. best effort compliance
2. strict compliance

"best effort" does not provide any interop guarantees, since it weakens conformance and leaves many implemmentation choices. Let's assume some mandatory fields would be missing. One consumer (A) could just reject the TD, another one (B)would be implementing fallback behavior. A 3rd consumer (C) would implement a different fallback, which has wrong behavior. Where is the interoperability guarantee? Who is taking responsibility for any misbehavior in consumer B or C? The device manufacturer, consumer manufacturer? Who is solving the problem? Who is paying a refund to the customer?

Strict compliance can provide these guarantees, either a device is compliant or not. There's no room for Pareto, i.e. no 80%:20% compliance.

Without strict compliance there won't be an ecosystem of device and cloud manufacturers who adopt WoT.

[egekorkan]

I agree with the motivation behind. For maximum interoperability one would need such requirements to be fulfilled and not have room for flexibility. The major problem is that we do not have a certification program or something similar. So even when a TD would validate a JSON Schema validation, we will never be able to say that the behaviour of the Thing is as the spec says. Thus, I would prefer to keep the validation strict but define a normative fallback mechanism as well.

[benfrancis]

I agree with @egekorkan. WoT is not like technologies like Wi-Fi, Bluetooth, ZigBee and HomeKit which have trademark-protected formal conformance testing. That's not how W3C recommendations work.

A WoT Consumer can not assume that a Thing which claims to conform to a profile actually conforms to that profile. But refusing to communicate with a Thing altogether because of something minor like a missing description is also not good for interoperability.

A Consumer needs to attempt to communicate with a Thing as if it conforms to the profile to which it claims to conform, but degrade gracefully as much as possible when it gets an unexpected result.

For example:

1. If a title is missing, use the affordance name instead
2. If a description is not available in the user's chosen language, fall back to the default language. If no description is provided at all, then don't display one.
3. Accept any kind of URI as an ID
4. If a title or description is longer than can be rendered on a given user interface, then truncate it and display an ellipsis to show the user it has been truncated
5. Support both strings and arrays of strings for particular members of the Thing Description
6. If multiple Forms are provided for the same operation, then a Consumer can pick the form it wants to used based on its own preferred protocols/sub-protocols/data types etc. If two Forms are equally useful then just pick one.
7. If a given interaction affordance doesn't provide a Form using a protocol the Consumer supports, then don't display that interaction affordance in the UI, or mark it as unavailable
8. If an interaction affordance responds in an unexpected way, show an error to the user to tell them this, but allow the user to try again or to perform other interactions

[mlagally]

I think it is good to consider and discuss the proposed fallback mechanisms, and also that there's no formal compliance certification. However I think we should make sure we don't allow too many gray areas and still have workarounds and fallbacks. These make life for all involved stakeholders more difficult.

If we have a schema that enforces the presence of certain keywords, these can be formally verified with a JSON schema validator. If somebody does not pass the schema validation, he is not profile compliant. It is as simple as that. Forcing consumers to implement all kinds of fallback mechanisms is just causing additional effort in the wrong place.

What prevents a developer of a TD to use a title? A description? Why not just provide the minimum set of metadata to ensure interop? These are trivial profile requirements, others are more difficult to satisfy, e.g. the action status behavior. Let's discuss in today's profile call.

[danielpeintner]

I don't have a strong opinion either way, but I would like to refer you to the endless discussions why HTML browser do accept and render broken HTML (event without letting the user know that something is broken).

I think in the end it boils down to that costumers using an interface wanna see a result. They are simply not interested in errors popping up. Hence, UI vendors will do its best to hide possible errors as long as possible. I know, from the technical point of view it is wrong but this will happen for profile TDs as well.

Having said that, a JSON Schema validation for developer purposes is fine. In reality there will be fallbacks implemented ...

[benfrancis]

@mlagally wrote:

What prevents a developer of a TD to use a title? A description?

In which languages? Even this is not necessarily trivial.

I think the important point is not whether the specification should require a title, but what a Consumer should do if a Thing doesn't provide a title (or a title in the user's preferred language). Simply saying that a term is mandatory doesn't provide enough information to implement a robust Consumer which can cope with real world conditions (where titles will inevitably be omitted).

Either the Consumer can still function without the title (in which case it isn't really mandatory), or Consumers must stop processing the TD altogether if a title is not provided (which is bad for interoperability).

This is why if we want to maximise interoperability we can only really recommend terms like title be present, and define fallbacks for when they are not.

Ideally I think these kinds of fallback mechanisms should be defined in the Thing Description specification, not left down to profiles. This is why I have proposed a deliverable for TD 2.0 which defines "an algorithm for parsing a Thing Description".

[mlagally]

We should revisit Validation in Profile-1.1