PROTO encoding

[sachinholla]

Target can format the PROTO encoded Update messages in multiple ways.

gNMI spec, section 2.3.3 asks to fill protobuf serialized data into TypedValue.any_val field using protobuf.Any message.

2.3.3 Protobuf Data encoded using the PROTOBUF type (i.e., within the any_val field) contains a serialised protobuf message using protobuf.Any. Note that in the case that the sender utilises this type, the receiver must understand the schema (and hence the type of protobuf message that is serialised) in order to decode the value

The protobuf-vals.md proposal allows to directly fill protobuf serialized data into TypedValue.protobuf_bytes field.

Encoding in TypedValue A new field is included within the TypedValue message's value oneof named protobuf_bytes. When a target or client populates this field it indicates that the content is a binary-marshalled Protobuf message. The protobuf message's type is determined by the tuple of <origin, Notification.prefix + Update.path>

The readymade Notification message formatter API ygot.TogNMINotifications creates an Update for each of the leaf data in a GoStruct; with value formatted as a scalar value (TypedValue field 1-8). I believe this also can be treated as PROTO encoding since it does not fit into any of the other encodings (bytes, ascii or json).

A target can support all of these modes when it uses yang based data model. However there is only one Encoding enum to denote the PROTO encoding. There is no way to pick a mode based on client's preference. This creates compatibility issues if target and clients (say, Network-OS and controllers) are developed by different parties. We can solve this through a compile time or runtime setting on either end; but it will be a non-standard solution.

Can we enhance the gNMI spec to standardize this PROTO encoding mode selection? Probably though new Encoding enums or a well-known extension.

[aashaikh]

As described in the specification, the client-specified encoding (PROTO, JSON_IETF, etc.) is intended for structured data carried in the TypedValue proto message. PROTO indicates that the structured data is carried as an out-of-band specified protobuf (using protobuf.Any).

The ygot helper function ygot.TogNMINotifications marshals Go struct fields into the corresponding TypedValue fields as scalar values per the recommended implementation for telemetry. PROTO encoding is not relevant here because it is not structured data.

The proto_bytes TypedValue is a further enhancement that provides a way to support proto-encoded structured data where the IDL is specified by a YANG model as described in the protobuf-vals.md doc.

I'm not sure what further standardization is needed here?

@gcsl may be able to comment further.

[sachinholla]

Thanks for the response @aashaikh. I did not realize that ygot.TogNMINotifications is meant for telemetry updates only. I do have a couple of follow up questions:

1) When you say "recommended implementation for telemetry", are you referring to following paragraph in section 3.5 of gNMI spec?

When aggregation is not permitted by the client or the schema each update message MUST contain a (key, value) pair - where the key MUST be a path to a single leaf element within the data tree (encoded according to Section 2.2.2). The value MUST encode only the value of the leaf specified. In most cases, this will be a scalar value (i.e., a JSON value if a JSON encoding is utilised)...

It is not very clear to me what the spec meant by "scalar value". Is it referring to scalar types as explained in section 2.3.3? The very next sentence says "a JSON value if a JSON encoding is utilised". I think ygot.TogNMINotifications encodes using scalar types and does not use JSON types. So, what is the recommendation for the implementers? Can we use scalar types for telemetry updates irrespective of the encoding requested in the SubscriptionList?

2) If target implements a YANG based data model, should it use TypedValue.protobuf_bytes or TypedValue.any_val for PROTO encoded structured data? I believe TypedValue.protobuf_bytes approach is the recommended one. But it is possible to encode the same through TypedValue.any_val as well. Should client and target agree on a specific mode out-of-band?

[sachinholla]

Original question was answered; closing..