Add APIs useful for data plane add-on-miss tables for PNA

[jafingerhut]

In the specification of the Portable NIC Architecture (PNA - https://github.com/p4lang/pna) there is a feature to declare a table as add-on-miss, meaning that if you apply() the table in your P4 program while processing a packet, and a miss occurs, the data plane can add a new entry with the key (exact match only) that was just missed, plus an action and its parameters.

Another new option is the ability of the data plane to delete old entries after they have not been matched for a configurable length of time, similar to the existing idle timeout feature, except that the control plane need not be notified, and the data plane will delete the entries without the control plane having to do so.

It is still desirable to allow the control plane to read entries of such a table. In a programmable NIC context, some tables would be so large that pagination APIs may become more important (see separate issue https://github.com/p4lang/p4runtime/issues/360 for that topic).

I have heard that some people implementing such add-on-miss tables would prefer to implement them in such a way that the control plane CANNOT add, modify, or delete entries, i.e. only the data plane could do so. It would be good to document that some implementers might not implement these operations on such tables. Even if a device implementer DOES implement these operations, note that it is possible for sequences like the following to occur, and control plane authors and P4Runtime API implementers would need to plan for these possibilities:

• data plane adds 10 entries
• control plane reads those 10 entries
• data plane adds an 11th entry with a key K11 different than the existing 10 entries above
• control plane attempts to add a new entry with K11
• P4Runtime server should reject the attempted add with some failure status code, because key K11 already exists. This is so, even though above the control plane read all existing table entries, and key K11 was not in there at that time.

Other possibly useful control plane APIs include:

• read the current number of entries, without reading the entries themselves
• configure a threshold number of entries K such that if the number of entries exceeds K, the control plane is notified
• perhaps a similar threshold on which a similar threshold of "if the number of entries goes below K", but that seems not quite as often useful as the previous one.
• perhaps the ability to configure the maximum number of data-plane added entries that can exist simultaneously, which could be used to 'reserve' some number of entries that can only be added by the control plane.

There may be other control plane API capabilities that would be useful to introduce here, too. The above is an initial list to get the discussion going. I will invite some other interested parties to add more to the list, if they have thought of any.

[jafingerhut]

Another possibility: reading large tables via a kind of pagination API, e.g. controller reads first 10,000 entries, then sends another request to get the next 10,000 entries, etc. This of course raises questions about what kinds of guarantees the read results represent (e.g. should they represent a consistent snapshot of the entire table at one point in time, or not? If not, are there any guarantees made at all?). EDIT: I created a separate issue https://github.com/p4lang/p4runtime/issues/360 for the pagination proposal, in hopes that the extensions raised on this issue can make more rapid progress independently of the pagination ideas, which I expect will involve quite a bit of thinking and discussion.

Note that I believe all of these proposals for extensions to P4Runtime API make sense to implement even for a target device that does not implement these new P4 tables where the data plane can add new entries without the controller requesting it.

[jafingerhut]

AI Andy: Ask Sayan if he is interested in collaborating on writing a proposal of specific changes to the protobuf messages for all of these API additions except the pagination one.

[jfingerh]

@jameschoi I will ping you on several public p4runtime issues to call your attention to them. For others, sorry for the comment noise.