Proposal: Adding Optional Extensions to gNMI

[robshakir]

Adding Optional Extensions to gNMI

Contributors: Rob Shakir (robjs@google.com), Carl Lebsack (csl@google.com), Nick Ethier (nethier@jive.com), Anees Shaikh (aashaikh@google.com)
Date: November 2017
Status: New Proposal

Summary

This contribution proposes to modify the gNMI protocol to allow extensions to be added without requiring the core specification to be modified. The intention of this is to allow implementors to add additional functionality to existing gNMI RPCs.

Problem Statement/Motivation

In particular use cases, there are requirements for data that is not currently specified within the existing RPC payloads to be carried. Some examples of this include:

• Implementations of gNMI which act as a proxy to set of downstream devices. In this case, there is a requirement to convey, along with each RPC, the target to which the request is being transmitted.
• Scenarios where the network element participates in arbitration between multiple writers. In this case, the payload of a Set RPC carries additional data that allows the target to determine whether the client is the most up-to-date writer, and hence determine whether to accept a Set request.
• Implementations of gNMI wherein the target wishes to supply additional (non-fatal) errors or warnings to the client in Get or Subscribe RPCs. For example, conveying that a warning condition occurred when retrieving the configuration (e.g., the device was not able to map a part of a native schema back to the requested schema).

The common requirement of these cases is that they require additional data to be supplied in addition to the existing payload without changing the RPC's semantics. If the semantics of an RPC are to be changed, defining an additional service (e.g., FooGNMIExtService with a new MagicSubscription RPC) provides a clean way to extend the service definition. However, for the cases above, where the semantics stay fundamentally the same, bifurcating RPC paths does not seem an optimal solution.

One proposed solution to this problem is to add additional fields in the gRPC metadata. There are a number of downsides to this approach:

• Logging, tracing or debugging frameworks must be extended to support logging gRPC metadata to allow the whole context of a request to be understood. Where logging frameworks simply take the serialised protobuf input as the logging payload, metadata is omitted.
• Semantically, since in some cases (e.g., that of the proxy) the information is really a first class requirement of the RPC payload, it seems fragile to rely on metadata for such information.

This proposal seeks to define a means by which existing gNMI RPC payloads can be extended using a non-metadata approach.

Proposal

We propose to introduce an additional repeated field to all top-level gNMI RPC messages (i.e., Capability(Request|Response), Get(Request|Response), Set(Request|Response) and and Subscribe(Request|Response)) named extension. This field is defined to a carry an gnmiext.Extension message.

We propose that the Extension message is defined as follows:

message Extension {
    oneof ext {
        ProxyDetails proxy = 1;
        ...
        RegisteredExtension ext = NN;
    }
}

message RegisteredExtension {
    enum ExtensionID {
        UNSET = 0;
        EXPERIMENTAL = 999;
    }
    ExtensionID id = 1;
    bytes msg = 2;
}

This splits gNMI extensions into two sets:

• Well-known extensions: These are defined to the set of extensions that are expected to be required by multiple implemenations. For example, numerous use cases have been described for carrying information that is useful to a target which acts as a proxy for other gNMI targets. For such common use cases, we propose to define messages directly within the gNMI extensions module.

The following example for a ProxyDetails extension is below.

// Credentials stores the authentication information for a proxy.
message Credentials {
  string username = 1;   // The login username.
  string password = 2;   // The authentication password.
  // TODO: Extend to other auth, e.g., SSL certificates.
}

message ProxyDetails {
  string address = 1;           // Address of the proxy.
  Credentials credentials = 2;  // Authentication credentials.
  proto.Any extra = 3;          // Additional (arbitrary) details.
}

• Registered Extensions: In other cases, extensions may be more esoteric, or required only in a subset of cases. In this case, we propose to register such extensions (particularly, to ensure that extensions do not make undefined modifications to the base behaviour expected of an RPC), but leave their payload defined by contrib definitions. We propose that an extension ID is allocated from the ExtensionID simply by requesting such an extension, and providing a link to a reference to the specification for the extension. The payload of the RegisteredExtension msg field is the marshalled contents of the message. The ExtensionID acts in the same manner as the type field of google.protobuf.Any, albeit with more restrictions as to the allowed set of types.

If a registered extension becomes popular, it may be promoted to a well-known extension. Initially, we propose that RegisteredExtension IDs are assigned simply through an email request. We do not propose to assign an extension ID to a well known extension. When extensions are promoted to well known, their extension ID will be deprecated over time.

Open Questions

• Currently, this proposal does not allow supported extensions to be discovered. The Capability RPC could be extended to add such discovery.

[robshakir]

@tsuna @jsterne @kuvempu, @nileshsimaria @aaronbee -- would be interested in your thoughts here.

[kuvempu]

Overall, useful to include. Some thoughts ...

• On error reporting: errors are reported per RPC already. Would prefer the Error message be tightened and extended (error code ranges for fatal and non-fatal errors, or add an error type as fatal or non-fatal to the message), rather than including in Extensions. This is required anyway since errors are also more specific (e.g. a specific Subscription in a SubscriptionList).

• Add an extension destination: proxy or target.

• Would suggest an extension type (mandatory vs. optional) in the Extension message. If a proxy or a target does not understand a mandatory extension, it must fail the request. Using well-known as mandatory is a catch-22 - an extension can become well-known only iff every implementation supports it.

Proxy-specific considerations:

• Can a proxy remove received extensions? ProxyDetails would be a candidate.

• Would suggest NOT allowing a proxy to add/modify extensions.

[tsuna]

LGTM.

Just curious: what's the advantage of registered extensions as opposed to using google.protobuf.Any? To support custom / third-party extensions that may not have protobuf payloads, or to still centrally oversee such extension through the assignment process of extension IDs?

@kuvempu why would you suggest to disallow the proxy to add/modify extensions? Or did you mean it's OK for the proxy to drop an extension it consumes for its own use (such as the ProxyDetails extension suggested above) but not OK to add/modify extensions? I'm sure there are cases where proxies will want to add extensions — for example with a dial-out setup (see #42) a reverse proxy might need to tack on information about where the message originally came from (effectively doing the opposite of ProxyDetails).

[robshakir]

Hi @tsuna -- apologies, have been travelling and missed responding to you.

We wanted to keep some form of curation of what is implemented as an extension vs. something that really requires a new RPC. For example, one could implement Subscribe with a SubscribeRequest that really has all the parameters one expects within an extension, rather than using the existing fields - whereas such new Subscribe semantics should probably be implemented with a different RPC such that it's clear to the consumer. Effectively, we erred this way because it seemed a clearer way to keep a standardised API through gNMI.

Let me know if this doesn't make sense!

r.

[robshakir]

@kuvempu, thanks for the feedback.

• We have a TODO to clarify error reporting, the Error message that is defined within gnmi.proto can be used as the pyaload in the status.proto Status message that is returned as part of the gRPC return status. I agree that we probably want to have some extensions here, but I don't think that this can be fatal vs. non-fatal -- since the case of non-fatal errors should not use a return code other than OK, and in the client library implementations this means that the error payload isn't populated.
• For the destination, it's not clear to me that we need to define this for all extensions generically. If there are cases whereby a particular extension needs to determine which system along the path should interpret the extension, then this can be in those particular extensions AFAICS?
• For 'mandatory' extensions -- it seems like we should just make these part of the base spec, if they are really mandatory. Is there a reason not to do this?

For the proxy specific messages, we'll define the semantics in a short doc explaining this extension, but (@gcsl, please keep me honest):

• A proxy can remove a ProxyDetails message such that chained proxies don't need to understand the index they are in the chain.
• A proxy can modify parameters, for example, it might not be possible for one proxy to resolve some ID of a subsequent proxy, therefore we'd propose to allow modifying the proxy details in the chain.

Thanks! r.