Profile Mechanism 2.0 - Githubissues

Questions About Profiles

During the review of WoT Profile 1.0, lots of fundamental questions have been asked about the nature of profiles.

What are profiles for?
Are profiles only a "subset" of TDs?
Should profiles be allowed to constrain the TD Information Model? (e.g. by making certain members mandatory, disallowed, or limited to a constrained set of values)
Should profiles be allowed to extend the TD Information Model? (e.g. by defining new terms like "longitude" and "serialNumber")
Should profiles be allowed to define protocol bindings for features that can't be described declaratively in a Thing Description? (e.g. an action queue with dynamically created resources)
Must all Things which implement a profile also be fully consumable by Consumers which don't implement the profile?
Should there be profiles specific to a particular use case (e.g. digital twins) or deployment model (e.g. cloud services)?

Current Profile Mechanism

The current profile mechanism definition simply says that "In order to conform with a profile, a Web Thing MUST conform with all the normative statements in the profile's specification." Because there are no constraints on what those normative statements can say, the answer to all of the above questions is essentially "anything goes". We can only debate on a profile-by-profile basis what a profile should and should not do. Because a profile is just identified by a URI, anyone could technically define their own extension profile and there are no restrictions on what those profiles can do.

Member Suggestions

A couple of suggestions have also recently been made by working group members:

Any extensions of the TD Information model should use the already defined mechanism for TD context extensions (i.e. JSON-LD contexts)
We should first define sub-protocols, then reference those sub-protocols from a profile

Profile Mechanism 2.0

Drawing from these ideas, I'd like to suggest an alternative approach to profiles (potentially targeted at WoT Profile 2.0) which could resolve some of these issues by more tightly constraining what profiles can define:

Rather than defining concrete protocol bindings in profiles, define sub-protocols in separate specifications which can be referenced using the subprotocol member of Forms
Rather than defining new vocabulary terms in the WoT Profile specification, define new (or re-use existing) external semantic contexts which can be referenced by the @context member of Thing Descriptions
At this point we may decide that's all that's needed. But if we decide that profiles are still needed, then profiles could be more narrowly defined as machine-readable semantic contexts which list a finite collection of other contexts (which may also include terms for the names of defined sub-protocols).

The answers to questions posed above would be:

What are profiles for? Profiles guarantee out-of-the-box interoperability between Things and Consumers which implement the same profile, by defining finite collections of semantic contexts which Things may use and Consumers must support.
Are profiles only a "subset" of TDs? No, but they can only extend the TD information model using TD context extensions.
Should profiles constrain the TD Information Model? They could, but only by referring to a TD context extension which overrides existing terms (I think this is possible by relying on the ordering of JSON-LD contexts).
Should profiles be allowed to extend the TD Information Model? Yes, by referring to external semantic contexts.
Should profiles be allowed to define protocol bindings for features that can't be described declaratively in a Thing Description? Yes, but by referring to external sub-protocols, the names of which are defined in a semantic context.
Must all Things which implement a profile also be fully consumable by Consumers which don't implement the profile? Yes, but they may ignore Forms which use sub-protocols they don't support (along with any other semantic annotations they don't recognise)
Should there be profiles specific to a particular use case (e.g. digital twins) or deployment model (e.g. cloud services)? Maybe, but they may share common sub-protocols and semantic contexts in order to allow some level of interoperability between profiles.

Specification Changes

What would this mean for the existing WoT Profile specification?

Turn the current protocol binding sections of the existing profiles into separate sub-protocol specifications
Define (or re-use existing) semantic contexts which define new vocabulary terms (e.g. sub-protocol names, geolocation metadata, units)
Re-define the profile mechanism to use @context instead of profile, and constrain profiles to being JSON-LD contexts, rather than an unconstrained set of normative assertions.

Advantages & Disadvantages

Advantages:

More tightly constrains what profiles can and can't define, by re-using the existing TD context extension mechanism.
Resolves the current awkward situation where protocol bindings can either be defined declaratively in Forms (using protocol binding templates) or concretely in Profiles (using protocol binding specifications). Protocol bindings would always be defined using Forms, but those Forms may reference re-usable out-of-band protocol definitions using the subprotocol member.
Is a bit more consistent with the way the WoT Discovery specification defines a sub-protocol for directories (the Directory Service API) using @context and Forms, rather than a separate profile member
Would remove the need for a dedicated profile member, since the existing @context member could be used instead.
Makes it more explicit that a Consumer should not attempt to consume a given Form if it doesn't support the sub-protocol it uses (e.g. because it defines behaviour like an asynchronous action queue which relies on a sub-protocol definition). This will help prevent runtime errors in generic Consumers which are trying to a consume a Thing which uses a profile that the Consumer does not implement.

Disadvantages:

Consumers which don't implement a profile may ignore Forms using HTTP sub-protocols they don't recognise, even if they would actually be able to consume a subset of functionality just by following the existing small set of HTTP defaults defined in the TD specification.
The "subprotocol" term has a clearly defined meaning for WebSockets, but is less well defined for other protocols. E.g. one of the suggested values is "sse", but the HTTP SSE Profile defines a protocol binding which builds on top of SSE. Is that then a sub-protocol of a sub-protocol?
Semantic contexts are constrained in what they can define, but unconstrained in that anyone can define their own, which could result in an explosion of profiles that could fragment the Web of Things. This could be mitigated by taking a "registry" approach to profiles and/or subprotocols in a similar way to the approach being proposed for protocol binding templates.
Using semantic contexts and sub-protocols as composable building blocks is arguably not as neat as a dedicated "profile" member and self-contained profile specifications.
A sub-protocol has to be referenced in every Form in a Thing Description, rather than just defining a profile once at the top level.

Examples

A simple example of using a sub-protocol defined via a semantic extension:

{
  "@context": [
    "https://www.w3.org/2022/wot/td/v1.1",
    "https://w3c.github.io/web-thing-protocol/v1",
  ],
  "properties": {
    "level": {
      "type": "number",
      "forms": [
        {
          "href": "https://example.com/things/lamp/properties/level",
          "subprotocol": "webthing"
         }
      ] 
    }
  }

}

A more complex example of profiles defined as semantic contexts, derived from Example 22 in the WoT Profile specification:

{
  "@context": [
    "https://www.w3.org/2022/wot/td/v1.1",
    "https://www.w3.org/2022/wot/profile/http-basic/v1",
    "https://www.w3.org/2022/wot/profile/http-sse/v1"
  ],
  "id": "https://mywebthingserver.com/things/lamp",
  "base": "https://mywebthingserver.com/things/lamp/",
  "title": "My Lamp",
  "description": "A web connected lamp",
  "securityDefinitions": {
    "oauth2": {
      "scheme": "oauth2",
      "flow": "code",
      "authorization": "https://mywebthingserver.com/oauth/authorize",
      "token": "https://mywebthingserver.com/oauth/token"
    }
  },
  "security": "oauth2",
  "properties": {
    "on": {
      "type": "boolean",
      "title": "On/Off",
      "description": "Whether the lamp is turned on",
      "forms": [
        {
           "href": "properties/on",
           "op": ["readproperty", "writeproperty"],
           "subprotocol": "wot-http-basic"
        },
        {
           "href": "properties/on",
           "op": ["observeproperty", "unobserveproperty"],
           "subprotocol": "wot-http-sse"
        }
      ]
    },
    "level" : {
      "type": "integer",
      "title": "Brightness",
      "description": "The level of light from 0-100",
      "unit": "percent",
      "minimum" : 0,
      "maximum" : 100,
      "forms": [
        {
           "href": "properties/level",
           "op": ["readproperty", "writeproperty"],
           "subprotocol": "wot-http-basic"
        },
        {
           "href": "properties/level",
           "op": ["observeproperty", "unobserveproperty"],
           "subprotocol": "wot-http-sse"
        }
      ]
    }
  },
  "actions": {
    "fade": {
      "title": "Fade",
      "description": "Fade the lamp to a given level",
      "input": {
        "type": "object",
        "properties": {
          "level": {
            "title": "Brightness",
            "type": "integer",
            "minimum": 0,
            "maximum": 100,
            "unit": "percent"
          },
          "duration": {
            "title": "Duration",
            "type": "integer",
            "minimum": 0,
            "unit": "milliseconds"
          }
        }
      },
      "forms": [
        {
           "href": "actions/fade",
           "subprotocol": "wot-http-basic"
        }
      ]
    }
  },
  "events": {
    "overheated": {
      "title": "Overheated",
      "data": {
        "type": "number",
        "unit": "degree celsius"
      },
      "description": "The lamp has exceeded its safe operating temperature",
      "forms": [
        {
           "href": "events/overheated",
           "subprotocol": "wot-http-sse"
        }
      ]
    }
  },
  "forms": [
    {
      "op": ["readallproperties", "writemultipleproperties"],
      "href": "properties",
      "subprotocol": "wot-http-basic"
    },
    {
      "op": ["observeallproperties", "unobserveallproperties"],
      "href": "properties",
      "subprotocol": "wot-http-sse"
    },
    {
      "op": "queryallactions",
      "href": "actions",
      "subprotocol": "wot-http-basic"
    },
    {
      "op": ["subscribeallevents", "unsubscribeallevents"],
      "href": "events",
      "subprotocol": "wot-http-sse"
    }
  ]
}

I agree with almost everything that is proposed. Just that I would propose to keep the profile keyword since that could allow consumers to quickly understand what possible restrictions will be and if a profile makes a term mandatory, the Consumers of the profile can be written in an easier way with less if checks and more assumptions.

Are profiles only a "subset" of TDs? No, but they can only extend the TD information model using TD context extensions.

Maybe it is just verbiage but I would still see profiles being a subset of TD which MAY use extension points provided by the TD specification (e.g., subprotocol,@context and other ways that may come in the future)

Must all Things which implement a profile also be fully consumable by Consumers which don't implement the profile? Yes, but they may ignore Forms which use sub-protocols they don't support (along with any other semantic annotations they don't recognise)

Advantages: ...

Makes it more explicit that a Consumer should not attempt to consume a given Form if it doesn't support the sub-protocol it uses (e.g. because it defines behaviour like an asynchronous action queue which relies on a sub-protocol definition). This will help prevent runtime errors in generic Consumers which are trying to a consume a Thing which uses a profile that the Consumer does not implement.

This needs to go into the TD specification.

Some more notes/questions on the examples you provided (i.e., first "simple example" using subprotocol). I am not a JSON-LD expert but to me it is difficult to "see" to which context "webthing" belongs to. Wouldn't it be better/clearer to use prefixes?

{
  "@context": {
    "https://www.w3.org/2022/wot/td/v1.1",
    "wt": "https://w3c.github.io/web-thing-protocol/v1"
  },
  "properties": {
    "level": {
      "type": "number",
      "forms": [
        {
          "href": "https://example.com/things/lamp/properties/level",
          "subprotocol": "wt:webthing"
         }
      ] 
    }
  }
}

Question to JSON-LD folks: Is this in the end the same if there a no name collisions?

@egekorkan wrote:

I would propose to keep the profile keyword since that could allow consumers to quickly understand what possible restrictions will be and if a profile makes a term mandatory, the Consumers of the profile can be written in an easier way with less if checks and more assumptions.

I like the sound of that, but what would would go in the profile member and what would go in the @context member and how would they be treated differently to each other? The idea here is to use the other more clearly defined extension points of @context and subprotocol so that profiles are more constrained in what they can do. How would the profile member be treated differently in 2.0 than it is in 1.1?

@danielpeintner wrote:

a Consumer should not attempt to consume a given Form if it doesn't support the sub-protocol it uses

This needs to go into the TD specification.

Yes, this is the kind of thing which could be clarified by a normative algorithm for parsing Thing Descriptions in TD 2.0. It's kind of implied in TD 1.x, but not explicitly stated.

Wouldn't it be better/clearer to use prefixes?

Is this in the end the same if there a no name collisions?

The syntax isn't quite right because you missed out some curly braces, but yes that is equivalent.

The downside of using prefixes in the @context member is is that Consumers have to support JSON-LD prefix expansion (which can get complicated) rather than just parsing the TD as JSON. But yes I agree that using prefixes makes it more obvious where the additional terms are coming from, and avoids collisions of terms where that's desired. If the prefixes were part of a profile defined as a JSON-LD context (like is currently done with the htv: and tm: prefixes in the TD context for example) then maybe those prefixes could safely be hard-coded into a plain JSON TD parser.

Rather than defining new vocabulary terms in the WoT Profile specification, define new (or re-use existing) external semantic contexts which can be referenced by the @context member of Thing Descriptions

This is a common signaling mechanism, although not really "semantic". Technically, the context just defines terms and a couple of other things which defines how to interpret terms. It would be possible to copy some of the terms from a context and use them to say the same things. An alternative would be to declare the profile using an @type definition on a top-level object, for example. However, for the purposes of your profiles, this is a reasonable approach.

Another idea about the design of profiles has come out of a discussion regarding the definition of the term "protocol binding" in the next WoT Workng Group Charter. This is because it has been essentially suggested that the term "protocol binding template" be renamed to "protocol binding" throughout W3C WoT specifications. This is potentially confusing because it could result in there being an "HTTP Protocol Binding" document as part of the Binding Templates work, but currently WoT Profile also defines an HTTP protocol binding. Profiles currently provide an alternative prescriptive mechanism for defining protocol bindings for greenfield applications instead of the descriptive mechanism of protocol binding templates for brownfield applications.

One proposed solution to this conflict is to move the content of the "Protocol Binding" sections of profiles into a Protocol Binding document (e.g. "WoT HTTP Protocol Binding") and a Payload Binding document (e.g. "WoT JSON Payload Binding"), linked from the WoT Binding Templates registry. All of the prescriptive parts of the current protocol binding sections of profiles would become defaults in protocol and payload binding documents which all Consumers implementing those specifications would then have to support. Things could then opt-out of these defaults by overriding the protocol binding using the vocabularies provided in the binding documents. This would effectively combine the concepts of descriptive vs. prescriptive protocol bindings, where the latter is just a set of defaults for the former.

This would then also allow a simpler approach to profiles where they just reference protocol binding documents and payload binding documents instead of defining a protocol binding inside the profile.

The general approach would be that for each aspect of a Thing for which there are multiple implementation choices, a profile would prescribe a particular choice, e.g.:

Protocol binding - e.g. Things MUST use the HTTP Protocol Binding and conform to its defaults (i.e. not override them). Consumers MUST support the HTTP Protocol Binding where defaults are used.
Payload binding - e.g. Things MUST use the JSON Payload Binding and its default payload formats for particular operations. Consumers MUST support the JSON Payload Binding and default payload formats for operations.
Security mechanisms - e.g. Things MUST use at least one of the NoSecurityScheme, BasicSecurityScheme and OAuth2Scheme. Consumers MUST support all of the NoSecurityScheme, BasicSecurityScheme and OAuth2Scheme.
Discovery mechanisms - e.g. Things MUST be discoverable at an HTTP URL using a Direct Introduction Mechanism. Consumers MUST be able to retrieve a Thing Description from an HTTP URL discovered using a Direct Introduction Mechanism.
Link relations - e.g. Consumers MUST interpret rel=icon links as an icon for a Thing.
Semantic contexts - e.g. Consumers MUST support a particular semantic context for units or geolocation.

Example profiles:

HTTP+JSON Profile - Uses the HTTP Protocol Binding and JSON Payload Binding
HTTP SSE+Event Stream Profile - Uses the HTTP SSE Protocol Binding and Event Stream Payload Binding
HTTP Webhook+JSON Profile - Uses the Webhook Protocol Binding and JSON Payload Binding (?)
CoAP+CBOR Profile - Uses the CoAP Protocol Binding and CBOR Payload Binding

I would suggest we should try to keep the list of profiles as small as possible in order to maximise plug-and-play interoperability between greenfield implementations and not cause fragmentation.

This approach would require that the task force working on Binding Templates are willing to accept the content from the current protocol binding sections of profiles into protocol binding and payload binding documents.

The big advantage of this approach vs. using the subprotocol member of forms to point to a sub-protocol specification (suggested above) is that Consumers would support the protocol binding by default, whereas with the sub-protocol approach Consumers which don't explicitly implement a profile may just ignore Forms which use a sub-protocol they don't recognise.

CC @egekorkan

To illustrate a bit more concretely how I imagine profiles and bindings could work together, below are example document outlines for an "HTTP Protocol Binding" specification, a "JSON Payload Binding" specification and an "HTTP+JSON Profile" specification which references them.

HTTP Protocol Binding

Introduction
HTTP Vocabulary
Default Protocol Binding [1]
1. Properties
  1. readproperty
  2. writeproperty
  3. readallproperties
  4. writeallproperties
2. Actions
  1. invokeaction
  2. queryaction
  3. cancelaction
  4. queryallactions

JSON Payload Binding

Introduction
Data Schema Serialisation [2]
Default Payload Formats [3]

HTTP+JSON Profile

Identifier
Protocol Binding [4]
Payload Binding [5]
Security
Discovery
Links
Semantic Contexts

Notes:

The default protocol binding would be derived from the Protocol Binding section of the HTTP Basic Profile in Profile 1.0 and would go beyond basic default verbs to define other details like HTTP headers, response codes etc.
The data schema serialisation would be very straightforward for JSON since how to map JSON Schemas onto JSON is already well defined, but it would be more involved for other payload bindings
The default payload formats would be trivial for most operations since it would be a direct mapping from a JSON schema in a TD, but it could more explicitly define the payload format of meta operations like readallproperties, readmultipleproperties, writemultipleproperties, writeallproperties, and crucially define a default payload format for queryaction operations.
The protocol binding section of the HTTP+JSON profile would assert that conformant Things and Consumers MUST implement the Default Protocol Binding from the HTTP Protocol Binding document
The payload binding section of the HTTP+JSON profile would assert that conformant Things and Consumers MUST use the default payload binding formats from the JSON Payload Binding

This would be a big change for Consumers, because all Consumers which implement a given binding would have to implement support for its default protocol binding (providing more interoperability by default), but it would also mean that Consumers which only care about Things which conform with a particular Profile would only have to implement the defaults used by that profile, and not support the full protocol binding vocabulary (which is much more complex to implement).

Edit: There are some details I'm not sure about at the intersection of the protocol and payload bindings where the two currently work together in the HTTP Basic Profile. E.g. Accept and Content-Type headers being set to application/json in HTTP, and HTTP URLs being provided in JSON payloads like the ActionStatus object.

I think we should discuss a relationship with Binding Template TF and Profile TF. But I don't think any new specifications of binding template should be described at the profile document. If we want to describe some new specifications of binding template, we should propose them to Binding Template TF. Because if multiple documents have different binding template specifications, it will be difficult to maintain consistency, and it will be difficult for engineers who use it to understand. So the binding template specification should be described in the Binding Template documentation, not the Profile documentation.

@chachamimm wrote:

I think we should discuss a relationship with Binding Template TF and Profile TF. But I don't think any new specifications of binding template should be described at the profile document. If we want to describe some new specifications of binding template, we should propose them to Binding Template TF.

Yes, as I said in https://github.com/w3c/wot-profile/issues/285#issuecomment-1433087304:

This approach would require that the task force working on Binding Templates are willing to accept the content from the current protocol binding sections of profiles into protocol binding and payload binding documents.

Please see https://github.com/w3c/wot-binding-templates/issues/248 for that discussion.

To be clear, I am not proposing that "binding templates" would be defined in the Profile specification, but that the Profile specification would reference and depend upon default bindings defined in the protocol and payload binding documents already proposed for the next charter.

I have created a WoT HTTP Basic Profile 2.0 (and companion WoT HTTP Protocol Binding 2.0) strawman proposal to explore the ideas discussed in this issue in more detail.

See my email on the WoT WG mailing list for more explanation.

Feedback is welcome here.

I like the suggested direction, but I'd like to have the TD 2.0 itself see an overhaul first since it might make some of the profile and bindings sections simpler. Also we should put somewhere a section on what's the expectation regarding degraded consumption.

Note: Maybe it is also good to share some ideas/proposals to

what is the relation to WoT Profile 1.0?
Are there any updates to Profile 1.0 planned?
What is the timeline for 2.0 ?
...

@lu-zero wrote:

I'd like to have the TD 2.0 itself see an overhaul first since it might make some of the profile and bindings sections simpler.

I agree, but I think those activities are going to have to happen in parallel since they are so inter-dependent. It may also make sense to move some assertions (e.g. section 8.3.1 Protocol Binding based on HTTP) out of the Thing Description specification as part of that process.

Also we should put somewhere a section on what's the expectation regarding degraded consumption.

What is degraded consumption?

What is the expectation on the consumer that may have no full knowledge of what is in the thing being consumed e.g.:

What to do if an affordance has data schemas with unknown vocabulary terms
What to do if a unknown term is seen in the affordance itself

Sometimes you have to completely omit the affordance, sometimes can just omit the form and fallback to known ones.

@danielpeintner wrote:

Note: Maybe it is also good to share some ideas/proposals to

what is the relation to WoT Profile 1.0?

Are there any updates to Profile 1.0 planned?

What is the timeline for 2.0 ?

These are good questions, and currently I would say it's mainly a case of resources. Oracle have stepped back and I have been told that Profiles are not currently a priority for Siemens.

Personally I would like to continue to be an editor of WoT Profiles 1.0 and get it published as a Recommendation, because it's going to be a long time until the WoT 2.0 specifications are published and I believe something is needed in the meantime. Currently I am trying to find a source of funding to support that work, and I can't do it alone. (I understand that @lu-zero would like to contribute, which is great).

Given the lack of implementations of WoT Profiles 1.0 I also think there's an argument for just publishing the current version as a W3C Note, and moving onto a refined design for WoT Profiles 2.0 which may attract more implementations. I think there's a general agreement that Profiles are needed, but I sense there are some reservations (see my original post) about the 1.0 approach (necessitated by limitations of the TD 1.x specifications).

For the record, my answers would be:

Profiles 2.0 is a backwards incompatible replacement for Profiles 1.0
I don't expect there to be a WoT Profiles 1.1
2.0 can start when 1.0 is finished (whatever form that takes)

@lu-zero wrote:

What to do if an affordance has data schemas with unknown vocabulary terms

What to do if a unknown term is seen in the affordance itself

These seem like general WoT questions not specific to Profiles, but I would say that if a Consumer encounters vocabulary terms using a context extension it doesn't implement then it should ignore them.

In general I expect Consumers to choose a Form which uses a protocol binding it implements, and ignore affordances which don't have any Forms it can use.

I also think it's reasonable for Consumers to only support Things which conform to a particular profile, and to ignore individual affordances which fall out of the scope of that profile (there's a note to this effect in the existing spec and my strawman proposal, but we could be more explicit about what can safely be ignored).

@lu-zero wrote:

What to do if an affordance has data schemas with unknown vocabulary terms

What to do if a unknown term is seen in the affordance itself

These seem like general WoT questions not specific to Profiles, but I would say that if a Consumer encounters vocabulary terms using a context extension it doesn't implement then it should ignore them.

Yes we need to write it at least in TD 2.0.

In general I expect Consumers to choose a Form which uses a protocol binding it implements, and ignore affordances which don't have any Forms it can use.

This seems to be the consensus to me as well, there are corner cases when a protocol binding wants to insert terms within the data schemas, in that case we might have terms fine to ignore or something that makes the whole affordance invalid.

I also think it's reasonable for Consumers to only support Things which conform to a particular profile, and to ignore individual affordances which fall out of the scope of that profile (there's a note to this effect in the existing spec and my strawman proposal, but we could be more explicit about what can safely be ignored).

That would be one of the best selling points for profiles: if a consumer supports a profile and a thing signals a profile they would fully interoperate within that profile, a consumer understanding only part of the context cannot ensure it can consume even minimally the thing w/out trying.

w3c / wot-profile

Profile Mechanism 2.0 #285

HTTP Protocol Binding

JSON Payload Binding

HTTP+JSON Profile