Indicating the relevant part of a payload

[egekorkan]

Observation

In some platforms or standards, a certain part of the payload is the actually relevant part for the application/mashup logic. For example, I want to read the temperature of a room and a sensor can provide this to me. Then I want to use this value to toggle a heater. In some cases, the readproperty operation on a property affordance called temperature results with a payload like this:

{
  "temperature": 12
}

It is important to note that the payload of the temperature affordance contains again the word temperature as a JSON object key. Below are some other possibilities that shows that the affordance name does not necessarily match the key in the payload:

{"temp": 12}	{"value": 12}

The examples above are abstract but below are concrete examples from known platforms and standards.

ECHONET Lite Web API

Executing a readproperty operation on the property called rgb, returns the following:

{
  "rgb":{
    "r":253,
    "g":12,
    "b":45
  }
}

Source: ECHONET Consortium presentation on the WoT Open Day on 14.10.2021 The TDs at https://github.com/w3c/wot-testing/blob/main/events/2021.09.Online/TD/TDs/ECHONET/generallighting.td.jsonld are results of the intermediary that remove the top layer.

Here, the relevant payload is the value of /rgb. This uses the JSON Pointer notation.

Philips Hue

Executing a readproperty operation on the property called lightInformation, returns the following:

{
    "state": {
        "hue": 50000,
        "on": true,
        "effect": "none",
        "alert": "none",
        "bri": 200,
        "sat": 200,
        "ct": 500,
        "xy": [0.5, 0.5],
        "reachable": true,
        "colormode": "hs"
    },
    "type": "Living Colors",
    "name": "LC 1",
    "modelid": "LC0015",
    "swversion": "1.0.3"
}

Note: In some cases, the payload is even bigger. Please see here.

Source: https://github.com/w3c/wot-testing/blob/main/events/2021.09.Online/TD/TDs/TUM/philips-hue/lightTD1.td.jsonld

Here, the relevant payload is the value of /state. Of course, if I want to put different affordances for each of these like lightHue, lightBrightness, the relevant payload would be at /state/hue and /state/brightness respectively.

Possible Solutions

The above examples were using the HTTP protocol but basically, this can happen with any application or transport protocol.

Subprotocol

We can say that these are subprotocols. This would mean that the DataSchema of the property will not contain the top level structure, i.e. it will not contain rgb or state from the examples above. This would mean creating a different subprotocol specification for every platform or standard that does this.

My opinion: We should not do this. The above examples are not complicated subprotocols and this may result in everyone defining a subprotocol. This would greatly hamper interoperability. I would suggest putting the entire payload's schema rather than having different subprotocols.

Linking to relevant parts

I think that we can solve this by simply indicating what parts of the payload are relevant for the application logic. This would mean putting the entire data schema in the TD and then indicating with a JSON Pointer on the relevant part. Example Property Affordance below:

{
    "@context": "...",
        ...
    "properties": {
        "brightness": {
            "type": "object",
            "properties": {
                "brightness": {
                    "type": "number",
                    "minimum": 0.0,
                    "maximum": 100.0
                },
                "timestamp": {
                    "type": "string",
                }
            },
                        "payloadRef": "/brightness",
            "forms": []
        }
    }
}

This would basically enable any WoT Consumer to understand the relevant part of the payload without relying on understanding an ontology or a subprotocol.

[mjkoster]

Or perhaps a payload schema in the binding form that links back to elements in the semantic schema:

{
  "@context" : [
    "http://www.w3.org/ns/td",
    { "odm": "https://onedm.org/onedm" }
  ],

  "properties": {
    "rgb": {
      "type": "object",
      "properties": {
        "BlueChannel": {"type": "number", "@type": "odm:/#/sdfObject/RGBColorControl/sdfProperty/BlueChannelData" },
        "GreenChannel": {"type": "number", "@type": "odm:/#/sdfObject/RGBColorControl/sdfProperty/GreenChannelData" },
        "RedChannel": {"type": "number", "@type": "odm:/#/sdfObject/RGBColorControl/sdfProperty/RedChannelData" }
      },

      "forms": {
        "href": "https://example.com/testdevice/rgb",
        "readable": true,
        "writable": true,
        "payloadschema": {
          "type": "array",
          "items": [
            {
              "@type": "#/properties/rgb/properties/RedChannel",
              "type": "integer",
              "minimum": 0,
              "maximum": 255
            },
            {
              "@type": "#/properties/rgb/properties/BlueChannel",
              "type": "integer",
              "minimum": 0,
              "maximum": 255
            },
            {
              "@type": "#/properties/rgb/properties/GreenChannel",
              "type": "integer",
              "minimum": 0,
              "maximum": 255
            }
          ]
        }
      }    
    }
  }
}

[k-toumura]

I agree with both (linking to relevant parts, and payload schema in the binding form) ideas. I think we should take a descriptive approach whenever possible.

[relu91]

As expressed in the comment linked right above, I like the idea of having payload information described in forms. I think it neatly describes what is just a payload format and what is application relevant. We should strive to maintain a clear separation between abstract DataSchemas (relevant to application logic) and binding communication information.

[miguelrk]

I like the idea of having the option of using JSON pointer notation to "describe" the incoming payloads. If this is indeed desired, I would say this would only get us half of the way though...

Using a JSON query syntax like JSON path would allow specifying which parts of the payload are application specific, but what about more complex transformations (get the last 3 values of an array) or even aggregations (e.g. average, minimum, maximum). I am thinking about a transformation? keyword, which allows specifying how to transform the incoming payload to a transformed (application logic relevant) value.

The transformation keyword can be express in a standard syntax like MathJSON, Jsonata , this could allow to do more than simply extract specific parts from the payload. This annotation could then be used by the consuming applications (in node-wot for example, this could be used internally by the .value() method).

This might be out of scope to include in the TD, but I would say it wouldn't harm it either. Then again, this keyword could be added with an external vocabulary and using the @context keyword already, if it is meant be consumed and handled by the consuming applications anyways, keeping the TD spec leaner.

[relu91]

Transformation capabilities are interesting to explore, I remember reading somewhere a use case where the device returned a tuple of two numbers [value, multiplier] and the real application relevant value of that property was actually value*multiplier. I am just a little bit worried about the computational burden that runtimes would need to support for this feature. Maybe not too much.

As you are suggesting I would start from a custom keyword (outside of our spec) and see how the community will respond.

[egekorkan]

By the way, this concept also applies to bitmasking. In Modbus devices, you can read a register which has multiple binary values inside where you select them via bitmasking. An example can be seen here on page 236 (sorry for german but hopefully it doesn't matter here), where you can check if a threshold value is passed. There are similar ones for alarms where one register contains multiple alarms where each bit shows a different alarm activation.

This brings the question of whether this mechanism should be in the binding level or whether we should indicate that the generic term we define can be used for bitmasking as well. I am leaning towards a generic term since even bitmasking is not specific to Modbus.

This also relates to restructuring uriVariables discussion we had before.

[egekorkan]

There is another related concept that is presented in an old WISHI meeting: https://github.com/t2trg/wishi/blob/master/slides/Binary%20data%20parsing%20and%20annotating%20with%20YOUPI%20-%20WISHI.pdf and also https://datatracker.ietf.org/doc/html/draft-petrov-t2trg-youpi-01