Extending P4 runtime RPC requests/responses

[adibrastegarnia]

I have a question about extending P4 runtime API. In gNMI there is an extension mechanism to extend top level request/responses gnmi Extension .The P4runtime spec says there is an extension mechanism but I am wondering how it can be used to do so if possible? What is the best way to add extensions to top level RPC messages?

[antoninbas]

Did you read the relevant section of the spec: https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html#sec-extending-p4runtime? I believe it's quite similar to how gNMI extensions work.

[adibrastegarnia]

We want to extend top level P4runtime RPC messages. In gNMI each of the top level messages has a list of gNMI extensions which is technically a protobuf type Any. For example, https://github.com/openconfig/gnmi/blob/master/proto/gnmi/gnmi.proto#L221 I just see these Any type messages in Stream RPC but not in Write or Read RPCs. What Am I missing?

[adibrastegarnia]

ExternEntry is defined part of the Entity. However it can be used to extend top level Read and Write requests but It doesn't seem right to me. Even we cannot extend SetForwardingPipelineConfigRequest. Unless I missed something obvious.

[antoninbas]

The ability to extend SetForwardingPipelineConfigRequest may not be very important: the p4_device_config field is already available to represent arbitrary target-specific binary data.

I can however see how the ability to extend existing entity messages, such as TableEntry, could be interesting.

If you have a specific proposal and some motivating use cases for it, please free to submit it here. The best thing to do would be to attend the next P4 API Working Group meeting, which is this Friday (08/05/2022) at 9:30AM PST, to discuss it. https://p4.org/working-groups/

[adibrastegarnia]

Our goal is to expose an extended version of P4 runtime API as NB of a control plane service to provide the same level of abstractions to applications and provides a controller platform with a minimum core functionality. From NB perspective of a controller, applications are dealing with multiple targets and controller acts like a proxy exposing P4runtime API. To achieve this goal, one of the required fields in each request is a TargetID (think about target ID as an ID given to a programmable entity in topology) that allows us to find appropriate connection for the given target and send the request to the target using P4runtime SB API. We achieved the same goal in the context of micro-onos and specifically onos-config subsystem onos-config with optional target field for each path + gnmi extensions to support multi-target/version transactions. However, both these protocols are designed to be a single target protocol by default but we are not limited to extend them to support multi target scenarios. I can prepare a proposal to present in one of the P4 API group meetings but perhaps not for this Friday (08/05/2022).

[smolkaj]

@adibrastegarnia what does NB stand for? And SB?

[smolkaj]

@adibrastegarnia P4Runtime has a notion of “device ID” — having only skimmed over your message, I wonder if that would suit your needs?

[adibrastegarnia]

SB (Southbound) API and NB (Northbound) API. Perhaps this https://github.com/onosproject/onos-config/blob/master/docs/README.md gives you a bit more context in terms NB and SB.

The problem is device ID is uint64 number and If I understand correctly it is something specific to the server. I found out it is the same number even for different targets (for example for stratum device id is always 1), which I guess somehow represnts the number of devices a P4runtime server controls . A target Id is a given name (a string) we assign to a programmable entity like P4 Switch (e.g. Switch-1, Switch-2). These IDs are defined by the user who is in charge of provisioning/discovering topologies in some way to identify each target uniquely.

[smolkaj]

Thanks for clarifying.

In the spec it says:

The device_id uniquely identifies the target P4 device

So my understanding is that it should be specific to the P4 target, not the server -- certainly that's how we use it at Google.

[adibrastegarnia]

The question is if device id is unique across the network? If you have two targets in the network can they carry the same device id?

[jonathan-dilorenzo]

I believe that's up to your usage. It should be unique to each target controlled by the same controller for sure. I guess you could theoretically have multiple controllers controlling different parts of the network and reuse device IDs, but probably you should just make them unique within the entire network (that you have control over) to make life easier.

[smolkaj]

It should be unique to each target controlled by the same controller for sure.

It should technically be sufficient for the device ID to be unique to each target controlled by the same P4Runtime server, no?

[adibrastegarnia]

I guess there are two views on this topic. If we consider P4runtime as a single target protocol then device id is limited to P4runtime server domain as I mentioned above and it should be unique in that domain. this is what I have seen so far. but from controller perspective, we can distinguish devices with human readable IDs that are unique across a control domain (a network) and this is nothing to do with device ID. in this case extending P4runtime API to support this case requires introducing a TargetID in string format which is nothing to do with device ID

[antoninbas]

It should technically be sufficient for the device ID to be unique to each target controlled by the same P4Runtime server, no?

Yes it's not mandated by the P4Runtime spec. But that's making things needlessly complicated for the controller IMO.

I guess there are two views on this topic. If we consider P4runtime as a single target protocol then device id is limited to P4runtime server domain as I mentioned above and it should be unique in that domain. this is what I have seen so far. but from controller perspective, we can distinguish devices with human readable IDs that are unique across a control domain (a network) and this is nothing to do with device ID. in this case extending P4runtime API to support this case requires introducing a TargetID in string format which is nothing to do with device ID

As pointed out by @smolkaj, the only strong requirement for P4Runtime is that the device ids do not collide for a given P4Runtime server (there could be one P4Runtime server per ASIC, per switch, per chassis, ... that's not mandated by the spec). That doesn't mean that you cannot impose stronger requirements for your deployment or for your controller implementation. Device id assignment is out-of-scope of P4Runtime.

You mentioned above that Stratum always uses device id 1. That's not correct. Some Stratum binaries may default to device id 1, but Stratum lets you choose any device id you want. There is an interesting (but high-level) thread on this topic: https://forum.p4.org/t/device-id-assignment-for-server-with-multiple-devices/240/6

In practice I would expect most P4Runtime users to use device ids which are unique within the control domain. It's easy to maintain a mapping from unique device id to unique human-readable device name in a controller.

Now if you want to have the ability to assign human-readable names to devices (in addition to device id), maybe for the sake of better logging or easier monitoring / debugging, that's a different topic. Personally, I'd still say this is out-of-scope of P4Runtime. Maybe this can be done with OpenConfig, but I'd defer to @bocon13 there.

TLDR; I disagree that any change is required in P4Runtime, at least not from a functionality perspective.

[adibrastegarnia]

Device Id was kind of a side topic. The reason that I opened this topic was how we can extend top level RPC messages like gNMI which doesn't seem supported for all RPCs such as read and write. I will defer this topic when I get a chance to bring it to the working group meeting.