Support for the gNMI extension field

[Ichabond]

gNMI extensions (https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-extensions.md) allow for extended features in the gRPC messages sent back. The extension field is present at the top-level of the gNMI RPCs. Juniper as a vendor is registering an extensionID for their telemetry: https://github.com/openconfig/gnmi/pull/76 Right now, their prototext looks like this:

extension: {
  registered_ext: {
    id: 1
    msg: "\n\x0cedge01.lis01\x10\xff\xff\x03\"\x0fsensor_1007_2_1*d/interfaces/interface[name='irb']/subinterfaces/subinterface[index='100']/state/counters/out-octets/2d/interfaces/interface[name='irb']/subinterfaces/subinterface[index='100']/state/counters/out-octets/:\x05mib2d@\x01H\xff\x89\x99\x97\x932P\xff\x89\x99\x97\x932`\x81\x8a\x99\x97\x932p\x01\x80\x01\x02"
  }
}

which contains a lot of helpful data, which should be exported as tags on the events.

The ask: Add generic support for the extension field, allowing for arbitrary protobuf files to be used to decode extension IDs (a config map of ID -> protobuf file maybe?). Then use the field in processor to generate tags from it.

The specific reason for this ask: This extension field contains linecard information of the linecard generating the response. This is necessary data to deduplicate certain events (logical interfaces spanning multiple linecards will generate an event per linecard, which will be clobbered, given they look like duplicates right now). If the extension data was exported as tags, it would be a unique event.

[Ichabond]

Specifically, this is an extension of the existing --proto-file support, but which is locked to the Nokia SROS protofiles.

I'll have a look if this could be done by just letting the user define a mapping of: extensionID -> protofile + proto root.

This would then get decoded properly, I think, given I'm seeing Unmarshall errors on the JSON side which are related to the binary-marshalled protobuf in the msg field inside extension. So I think if the code here: https://github.com/openconfig/gnmic/blob/main/pkg/api/target/subscribe.go#L290 is made more generic, at least decoding would work. From there, I think it being converted to tags would "just work"? Will try to get some PoC going to prove/disprove this understanding.

[Ichabond]

@karimra I have a working prototype. Before I make a PR though, a couple questions I'd appreciate your input on:

1. The current protobuf reflection is fo SROS-only, and relies on a fully compiled-in Protobuf that can decode whatever is in the Protobytes. As users might be relying on this feature for SROS, I would add it as a new distinct feature, specifically geared at the extenions field?
2. Configuration location: I'd add it as a subsection of the targets dictionary, given this is where the existing proto-parsing lives: https://github.com/openconfig/gnmic/blob/fb7904ebc35a65b3b50d06cfb27890f814eaa558/pkg/app/target.go#L138

[karimra]

1. The current protobuf reflection is fo SROS-only, and relies on a fully compiled-in Protobuf that can decode whatever is in the Protobytes. As users might be relying on this feature for SROS, I would add it as a new distinct feature, specifically geared at the extenions field?
2. Configuration location: I'd add it as a subsection of the targets dictionary, given this is where the existing proto-parsing lives: https://github.com/openconfig/gnmic/blob/fb7904ebc35a65b3b50d06cfb27890f814eaa558/pkg/app/target.go#L138

Yes for both, something like:

targets:
  router1:
    address: x.x.x.x
    #
    # other fields
    #
    registered-extensions:
      - id: 42
        proto-file: /path/to/protofile