jafingerhut
commented
5 years ago There are a few hints on this page regarding gRPC message ordering -- search for occurrences of the word "order": https://grpc.io/docs/guides/concepts.html
For example, bidirectional stream RPCs, of which P4Runtime's StreamChannel is one example, has the following description on that page:
"Bidirectional streaming RPCs where both sides send a sequence of messages using a read-write stream. The two streams operate independently, so clients and servers can read and write in whatever order they like: for example, the server could wait to receive all the client messages before writing its responses, or it could alternately read a message then write a message, or some other combination of reads and writes. The order of messages in each stream is preserved."
Is there a brief summary somewhere written up of the "queueing structure" of a P4Runtime gRPC session between a single controller and a one-device server?
For example, are all messages from server to client FIFO? Or are there multiple FIFO queues of messages, perhaps separated by message types, e.g. responses to table add messages are in one FIFO queue, but stream digest notifications are a separate FIFO queue from those? Or does everything go in one FIFO queue?
The answers to these questions have an effect on design of client software, e.g. does it need to consume one kind of message in a timely fashion, in order to prevent other types of messages from experiencing head-of-line blocking? Of course a client can consume received messages "quickly" in all cases, and internally create its own queues or other storage of such messages and process them in whatever order it might wish to. But knowing whether a client should do this is important to know.
For example: The answer to this question could easily affect how quickly a standby controller consumes and responds to a message from the server that it is now the master. If that gets behind a long queue of digest notification messages, or idle timeout notifications, and thus takes many tens of milliseconds for the standby controller to notice, that would be a poorly designed system. That is the responsibility of the controller software designer/developer, I know, and we can't stop them from creating bad designs. We can let them know briefly what would might lead to a good design, though.
I suspect the answer to this question is in the gRPC documentation, and thus is out of scope for this spec, but having a quick summary somewhere of this info would be useful to have, even if it is not in the P4Runtime specification document.