Simplified TD: forms

[mkovatsc]

Moderated focused discussion on the Simplified TD proposal and forms. Please stay on topic and use https://github.com/w3c/wot/issues/393 for any general remarks.

In https://github.com/w3c/wot/issues/393 @benfrancis wrote:

• "rel": ["get", "set"] does not match my understanding of how link relations are intended to be used. They are supposed to "describe how the current context is related to another resource", not what protocol methods can be used with that resource.
• http:SubProtocol also feels like a bit of a hack
• As discussed in the meeting, I'm still not convinced by this use of "forms" as a generic mechanism for declaratively defining protocol bindings to any protocol inside a Thing Description. I think we should continue to incubate an alternative approach (inside the Interest Group) of using concrete protocol binding specifications for a small number of web protocols.
• If multiple links are needed for the same capability, an array of link relations could be used ("rel":"self" could be omitted and assumed as a default, but is included here to show a more common use of a link relation):

  "level" : {
  "name": "Level",
  "@type": "LevelProperty",
  "schema": {
  "type": "number",
  "minimum": 0,
  "maximum": 100
  }      
  "links" : [
  {
  "rel": "self",
  "href": "/things/lamp/properties/level",
  "mediaType": "application/json"
  },
  {
  "rel": "self",
  "href": "coap://things.com/things/lamp/properties/level",
  "mediaType": "application/cbor"
  }
  ]
  }

[mkovatsc]

In forms, rel is not a link relation type. It is a form relation type that describes the operation, e.g., Coral:

A form can be viewed as an instruction of the form "To perform a form relation type operation on form context, make a request method request to submission IRI" where the payload of the request may be further described by form data.

Here get and set are not the methods, but identify that this form is to be used for getting the Property value and setting the Property value (maybe to big step, as they are currently readProperty and writeProperty). The methods are taken from the defaults and not shown in the TD, as they correspond to the default. For get forms it is GET, for set forms it is PUT.

The subProtocol is closing a gap in current links and forms. There is no way to express sub-protocols such as long-poll or SSE; today this is always hard-wired into the application, which is not applicable when you do not own and control both sides of the communication (cf. WebSocket purpose and problem for interoperability).

[mkovatsc]

In https://github.com/w3c/wot/issues/393 @benfrancis wrote:

This is an interesting reference which explains a lot of the thinking in your approach, thanks.

I'd be happy to see some incubation work to test out using this approach with IoT to prove that it allows for the kind of automatic ad-hoc interoperability you imply it does. I remain sceptical but am happy to be proven wrong. In the meantime I'd also like to continue to incubate an approach built only on links and concrete protocol bindings.

[mkovatsc]

The concept for hypermedia-driven application was for instance presented at the Internet Architecture Board (IAB) Workshop on IoT Semantic Interoperability in 2016 (paper). It was preceded and followed by more work in the IRTF of which you saw Coral by @ektrah, @mjkoster has HSML, @handrews JSON Hyper-Schema, and we followed up with joint work with the Multi-Agent Systems community in the International Workshop on the Web of Things Workshop. There is also complementary work on goal-driven service composition and automatically generated execution plans, e.g., RESTdesc

I can offer you a proof-of-concept implementation available on GitHub, where a hypermedia machine-client is able to do discovery by following roughly described link paths, get hypermedia controls from a device, and interact with the device whenever it knows the form relation type (and other a priori knowledge defined by REST); if the discovered device is replaced, the hypermedia-client can recover and continue working with the new device, even when it is a different model with different resources and different interaction flows. This is done by following links and submitting forms where client state is filled into requests, which thereby modify resource state on the device to reach desired goal (formally defined target application state). In that case, we used "CoRE-HAL" (warning, sloppy strawman), an extension of JSON Hypertext Application Language that not only has links, but also said forms. I am not sure if it will run out-of-the-box, as I lack time and good students to maintain and follow up on it. There are also more complex proofs of concept working, such as a hypermedia-driven version of the IEC 61851 standard, which defines the process how your electric car can charge at different charging stations. Please contact me directly, if you want to try out the iot-hypermedia PoC.

Note that the TD work is focused on the stable fundamentals of this: the semantic metadata, which also allows for automatic validation, machine-assisted engineering, and other concepts that are already in production here and there.

[mkovatsc]

Independent from that excursus, I also thought more on links vs forms in regard to Properties vs Actions. How about the following, which I like because it brings WoT back in line with my hypermedia ideas:

We restrict Properties to only have links, so they are like simple Web resources that allow GET and PUT, and they are identified by a URI and fully described by a link with the implicit, but apparently general GET/PUT assumptions of Web links. The interaction with Properties must follow this assumption, which should be made explicit in the TD spec: GET to read the value, PUT to write the value; CoAP has Observe to observe and let's say we can figure out how to do observe for HTTP in another thread. That means there is motivation for everyone to converge to a sensible default. Yet we do not drop the goal to be able to describe existing devices and IoT services as well! They just have to pay a higher price, which is the incentive to converge: they need to use Actions to do non-standard operations on Properties -- let's assume everyone at least uses GET to read. If not, the have to go 100% RPC and only use Actions, no Properties at all.

Actions have forms -- are forms? Let's see. Anyhow, invoking Actions correspond to sending those requests that modify resource state through something other than a PUT where the request body is a valid representation for the target resource (same schema...). Actions usually use POST, but can also be PATCH or even FETCH. They correspond to HTML forms, which do have action instead of href. I hope you see the similarity and where this is going. Since we need to specify the schema for the request body as well as the schema for the "action results" (the response body), they are already different from Properties, where we only have the schema of the target resource. I hope we can find agreement that there should be metadata in Actions that can describe the request method expected by the server/Thing. I still would call this a form, hence give Actions a forms element.

Events would be similar again to Properties in the following sense: they have just one schema, which defines the event data, which however is not a resource representation, because it is not state as in state transfer. Still Events can have a link element that points to an Event source, e.g., Event Source as in Server-Sent Events, a plain WebSocket that only pushes Event data for a single Event, a long-poll enabled HTTP resource, or something more complex where multiple (e.g., WebSocket) connections can be merged into a single open connection by using for instance a proper WebSocket sub-protocol. And yes, that could also work for MQTT and other PubSub transfer protocols.

[handrews]

We restrict Properties to only have links, so they are basically Web resources, as they are identified by a URI. The interaction with Properties must follow the de facto standards

Forgive the no doubt naive question as I'm struggling to catch up to all of the discussion threads and proposals, but given the various collisions around the term "properties" noted elsewhere, has there been any thought of calling them Resources instead? It sounds like that might even suggest the correct behavior?

[handrews]

Events would be similar again to Properties: they have just one schema, which defines the event data, which is not a resource representation, because it is not state as in state transfer.

But is it representing the post-state-change state? This is how I've done events in the past: they are either state change (in which case they are representations) or errors (in which case they are identical to the error payload that would have been sent synchronously). The identification of what sort of event is is is done through the URI.

[mkovatsc]

Okay, I have to relative further, as I just wanted to give helping comparisons and not imply equality.

Properties, Actions, and Events are interactions that are somewhat more from a programmer's views than a networking/communications engineer and help to describe how interactions can be performed on something as abstract as a "resource". I hope we can stick with this fundamental working assumption of Properties, Action, Events for WoT, as they are much better than RPC, but guide you to a useful resource design.

Properties are only like resources that directly expose state, so behavior usually attached to Web resources that allow GET and PUT. Note that Observing is basically just an optimization of polling with GET, which is why CoAP uses an option applied to a GET request instead of its own method. Observe notifications being similar to a GET response contain the post-change state. Observe is made for synchronizing state between server and client with eventual consistency.

Action are like resources that accept some request body and have internal rules how to deal with it, and/or may return content in action results (those response bodies that are not resource representations), so behavior usually attached to Web resources that allow POST.

Yet there are more methods such as FETCH in CoAP, which is safe and idempotent like GET, but takes a request body and may also return content, which is not the resource representation (but usually a representation filtered through the request body); so it es more like an Action. PATCH could be argued to work as partial write on Properties, but the exact handling depends on the patch document (a degree of freedom that is not described by links, similar to the method), so it is less transparent; with not much thought, I would put it also into the Action bucket.

WoT Events should not contain the post-change state of of a resource, because then they would be similar to observe notifications (cf. observable Properties). Events come from PubSub and are a counter-part of the state paradigm in REST. Events represent the state transition itself: a purchase or a buttonpress -- instead of an extended list of purchases and unpressed/pressed states, resp. In WoT, Events should come from state changes of non-exposed state (so no Property) and can help people who come from the PubSub world. Of course something else can be shoehorned in here such as an MQTT topic that pushes post-change state that is also available in a Property. Maybe people want that; let's see.

(Sidenote: Mapping events on REST is often done wrongly or at least for CoAP Observe with its eventual consistency (intermediate representations may be lost). Instead of unpressed/pressed, a better state-based model would be the counter of how often something was pressed or the time of the last press (door bell...).)

I hope this or that helps if more information is needed on this state-event duality...

[handrews]

@mkovatsc thank you so much for your patience and taking the time to write this up. This helps focus things a great deal. I've been trying to read through specs and RFCs (I actually read CoAP Observe yesterday but not all the implications sunk in) but I've had trouble wrapping my head around the motivations that are coming from outside of the "purist" REST aspect of things. I'll digest this some and comment further when I can.

One reason I am so interested in seeing how Hyper-Schema can (or can't) align with this project is that it's one of the most compelling real-world examples of both hypermedia and semantic web technologies working together. If I can't productively fit Hyper-Schema into the WoT model (conceptually if not by exact syntax), then I suspect I have more work to do on Hyper-Schema. Preferably soon while it's still quite malleable due to few implementations.

[benfrancis]

@mkovatsc

The subProtocol is closing a gap in current links and forms. There is no way to express sub-protocols such as long-poll or SSE; today this is always hard-wired into the application, which is not applicable when you do not own and control both sides of the communication (cf. WebSocket purpose and problem for interoperability).

Quick note: WebSockets do have a mechanism for negotiating a sub-protocol to use https://tools.ietf.org/html/rfc6455#page-12

[benfrancis]

@mkovatsc wrote:

We restrict Properties to only have links, so they are like simple Web resources that allow GET and PUT, and they are identified by a URI and fully described by a link with the implicit, but apparently general GET/PUT assumptions of Web links.

Sounds good.

Actions have forms -- are forms? Let's see. Anyhow, invoking Actions correspond to sending those requests that modify resource state through something other than a PUT where the request body is a valid representation for the target resource (same schema...). Actions usually use POST, but can also be PATCH or even FETCH. They correspond to HTML forms, which do have action instead of href. I hope you see the similarity and where this is going.

I understand what you mean. A POST to create an action request resource makes sense. A PATCH as a partial update to a property resource makes sense, but not sure I understand its use in forms to request actions. It could perhaps be used to modify an action request after it has been created (e.g. to adjust a target value while the action is still in progress or hasn't yet started).

I'm still not sure that forms are actually necessary vs. just a link to an action resource which acts as an action request queue, but they do at least make more sense for actions than for properties and events.

Since we need to specify the schema for the request body as well as the schema for the "action results" (the response body), they are already different from Properties, where we only have the schema of the target resource.

I haven't yet come across use cases for actions which return data, but I can certainly imagine they exist. Note that the response to an action request may just be a 201 CREATED with a URL for an action request resource, the 201 response wouldn't contain any output of the action because it may not have completed yet. You can instead GET an action request resource to get its current status (or status changes may be pushed down to the client using a push mechanism). The action request resource may eventually contain data about its outcome, once the action it represents has completed.

Events would be similar again to Properties in the following sense: they have just one schema, which defines the event data, which however is not a resource representation, because it is not state as in state transfer.

I agree that events can have a single schema to define the event payload.

Note that in Mozilla's REST API a GET on an event resource returns a list of recent instances of the event. This means that you have the option of polling the event resource to get new events, in addition to alternative push mechanisms like the event message over a WebSocket.

[mkovatsc]

This discussion has ended with the same issue as in https://github.com/w3c/wot-thing-description/issues/179: should we have hypermedia as "links and forms side-by-side" or "links with operations on top". Let's close this here and continue in https://github.com/w3c/wot-thing-description/issues/179