Create with client-assigned Instance ID over SenML JSON/CBOR

[kFYatek]

In LwM2M 1.1 TS, section 6.3.6. Create states that:

When there is no reference to Object Instance in the TLV/CBOR/JSON payload of the "Create" command, the LwM2M Client MUST assigns the ID of the created Object Instance.

While doing this is relatively simple using TLV, the documentation for the SenML JSON/CBOR formats seem to force referencing the Object Instance. In particular, section 7.4.4. JSON states that:

Each entry of the JSON format is a Resource Instance, where the Name attribute need to be prepended by the Base Name attribute to form the unique identifier of this Resource instance.

The Name attribute in an entry is a URI path relative to the Base Name; namely the Name attribute could simply be the full URI path of a requested Resource Instance when Base Name is absent.

[...]

In particular, when Base Name is set to the LwM2M Object root (e.g. "/"), the JSON format may support to return a hierarchy of Object Instances when Object Link datatype resources are reported (example given below).

This is further reiterated in Table 7.4.4.-1 (emphasis added):

Base Name - The base name string which is prepended to the Name value of the entry for forming a globally unique identifier for the resource.

Name - The Name value is prepended by the Base Name value to form the name of the resource instance. The resulting name uniquely identifies the Resource Instance from all others. Example:

• [...]
• When Base Name is not present, the Array entry Name is the full URI of the requested Resource Instance

(Side note: the usage of the term "Resource Instance" is imprecise, as Single-Instance Resources do not have Resource Instances; this wording might be worth updating as well)

All JSON and CBOR examples throughout the TS are constructed in such a way that concatenated Base Name + Name strings always form a complete data model URI, originating in the root of the LwM2M data model tree.

This makes the format which the SenML payload to the Create operation shall follow if it does not want to specify the Instance ID, unclear. It needs to be clearly specified in the TS.

Solution proposals

I am submitting two (mutually exclusive) ideas for solving the problem. Implementing either of them in the spec would solve this issue.

Proposal A

• If the concatenated Base Name + Name strings of a SenML JSON/CBOR payload form a path that does not start with a slash ("/"), the path is treated as relative to the Path specified in CoAP options.
• As a special exception, in the Create operation, such relative paths are treated as relative to the newly created Object Instance, whose ID is assigned by the Client.
• Mixing absolute and relative paths in a Create operation payload is not allowed; upon receiving such payload, the Client MUST respond with 4.00 Bad Request.

Example A.1: Create a new instance of the "APN connection profile" object, with resources 0 and 4 set and instance ID assigned by the Client:

CoAP POST Uri-Path:"/11" Content-Format:application/senml+json
Payload:
[
  { "n": "0", "vs": "New Profile" },
  { "n": "4", "v": 3 }
]

Example A.2: Create a new instance of the "APN connection profile" object, with resources 0 and 4 set and instance ID assigned by the Server to 42:

CoAP POST Uri-Path:"/11" Content-Format:application/senml+json
Payload:
[
  { "bn": "/11/42/", "n": "0", "vs": "New Profile" },
  { "n": "4", "v": 3 }
]

Note: in example A.2, specifying the full path in either Base Name or Name is mandatory. Relative path "42/0" would be interpreted as Resource ID 42, Resource Instance 0.

Proposal B

• The concatenated Base Name + Name strings MUST always form a path that starts with a slash ("/") and represents a full, absolute URI of a Resource or Resource Instance.
• As a special exception, in the Create operation, the Instance ID might be substituted with an asterisk ("*") to signify a new Instance ID assigned by the Client.
• Mixing unambiguous paths and paths containing an asterisk is not allowed; upon receiving such payload, the Client MUST respond with 4.00 Bad Request.

Example B.1: Create a new instance of the "APN connection profile" object, with resources 0 and 4 set and instance ID assigned by the Client:

CoAP POST Uri-Path:"/11" Content-Format:application/senml+json
Payload:
[
  { "bn": "/11/*/", "n": "0", "vs": "New Profile" },
  { "n": "4", "v": 3 }
]

Example B.2: Create a new instance of the "APN connection profile" object, with resources 0 and 4 set and instance ID assigned by the Server to 42:

CoAP POST Uri-Path:"/11" Content-Format:application/senml+json
Payload:
[
  { "bn": "/11/42/", "n": "0", "vs": "New Profile" },
  { "n": "4", "v": 3 }
]

Note: examples A.2 and B.2 are identical. In Proposal B, all paths MUST be absolute.

[mojanm]

Proposal A looks logical with a minor change to example A.2 where server assigns instance ID which should read as: CoAP POST Uri-Path:"/11/42/" Content-Format:application/senml+json Payload: [ { "n": "0", "vs": "New Profile" }, { "n": "4", "v": 3 } ] Proposal B uses wild card which is not supported in SenML.

[kFYatek]

CoAP POST Uri-Path:"/11/42/" Content-Format:application/senml+json

As specified by Table 6.4.4 in the Transport Bindings TS, POST on a path that refers to an Instance is mapped to the Write operation. Also, section 6.3.6 of the Core TS lists "Object ID" and "New value" as the only parameters to the Create operation. So your proposal would introduce an ambiguity in the CoAP mapping.

When using TLV as Create operation's payload format, the way to distinguish between those two cases is to either include the enclosing "Object Instance" (Type 00) header in the payload, or exclude it, having multiple "Resource Instance" (Type 01) or "Multiple resource" (Type 10) entries at the TLV payload's top level. My proposal is a more or less direct mapping of that idea to SenML.

[mojanm]

I see your point and agree. The change I previously proposed to A2 was primarily to address the issue of Object ID being repeated twice in the path as you’d shown in your example rather than anything else. So, the proposed changes should have been to remove object ID from “bn” field rather than getting rid of the “bn” field completely and moving the instance ID to URI path.

[kFYatek]

As far as I understand, what I proposed as example A.2 is compliant even with the current version of the spec - as I mentioned, the spec seems to imply that the concatenated Base Name + Name shall always be the full URI path of the Resource - which, yes, leads to some duplication with the Uri-Path, which is unfortunate, but cannot really be avoided.

So, the proposed changes should have been to remove object ID from “bn” field rather than getting rid of the “bn” field completely and moving the instance ID to URI path.

If I understand correctly, it would look like this:

CoAP POST Uri-Path:"/11" Content-Format:application/senml+json
Payload:
[
  { "bn": "42/", "n": "0", "vs": "New Profile" },
  { "n": "4", "v": 3 }
]

which, unfortunately, creates an ambiguity. The resulting path for the first entry is "42/0", and it is, in the general case, impossible to distinguish whether it is Resource 0 within Object Instance 42, or Resource Instance 0 within Resource 42 within unspecified Object Instance.

[sbertin-telular]

A slight modification to Proposal B would be to simply omit the instance ID. Here is an example based on Example B.1:

CoAP POST Uri-Path:"/11" Content-Format:application/senml+json
Payload:
[
  { "bn": "/11//", "n": "0", "vs": "New Profile" },
  { "n": "4", "v": 3 }
]

[sbernard31]

I read the specification again, If I didn't missed something, I think there is either a "missing example" to explain how we can do that with SenML-JSON or SenML-CBOR. Or

When there is no reference to Object Instance in the TLV/CBOR/JSON payload of the "Create" command, the LwM2M Client MUST assigns the ID of the created Object Instance.

or should be rewritten to say that only TLV is usable. (:warning: TLV is deprecated since LWM2M 1.1 for client)

@hannestschofenig, I think this affect LWM2M 1.2 too.

[sbernard31]

This is "fixed" in LWM2M v1.2 by excluding SenML-JSON and SenML-CBOR content format for this feature :

When there is no reference to an Object Instance in the TLV/LwM2M CBOR payload of the "Create" operation, the LwM2M Client MUST assign the ID of the created Object Instance.

(source : §6-3-6 Create-Operation)

I guess this mean we could consider this as a bug in LWM2M v1.1 ? Is this kind of bug will be fixed in previous LWM2M version or only the last version(currently 1.2) get "bug fixes" ?

[hannestschofenig]

This issue is only fixed in version 1.2 and not in earlier versions.

[sbernard31]

@hannestschofenig what is general advice for implementer of older version ?

For example, if I'm an implementer of LWM2M v1.0 or 1.1 and I face a specification bug that is only fixed in 1.2. I guess I should (if possible) implement the fixed behavior from 1.2 in my implementation of LWM2M 1.0 or 1.1 ?

[kFYatek]

@sbernard31 I'm not Hannes, but yes, that is the current consensus within the working group.

Releasing bugfixes for specifications of old protocol versions is not feasible at the moment. The working group is discussing options for addressing the problem of such bugfixes going forward, but at this point, we informally recommend consulting the newer spec versions as you described.

[sbernard31]

Releasing bugfixes for specifications of old protocol versions is not feasible at the moment. The working group is discussing options for addressing the problem of such bugfixes going forward,

That's would be great to have a solution for this.

we informally recommend consulting the newer spec versions as you described.

@kFYatek thx for clarify this :pray:.