antoninbas
commented
5 years ago I'll just provide some context here, but this an issue that is common to a lot of gRPC services.
By default gRPC limits message sizes to 4MB. In my case this is too small, because my SetForwardingPipelineConfig messages are usually larger than that, so I routinely increase the "max receive message size" at the server (https://github.com/p4lang/PI/blob/master/proto/server/pi_server.cpp#L565).
One important thing about this configuration parameter is it has to be set when building the server, and cannot be changed during its lifetime. It also impacts the entire gRPC server, not a specific service (e.g. P4Runtime). In practice I'm not sure how this would play out. It seems that when deploying a NOS which integrates a gRPC server, this should be available as a configuration parameter, potentially platform-dependent. The controller may be able to query that value, but has doesn't have any ability to modify the value. For write batches, the controller may be able to organize its batches so that they don't exceed the max size (which will probably always be >= 4MB, because there is no reason to modify the default size except to increase it...). For SetForwardingPipelineConfig, there is no possibility to split the message. However, because the config parameter can be platform-aware, we can ensure that it can accommodate all possible SetForwardingPipelineConfig messages. For example, I can come up with an upper-bound for all SetForwardingPipelineConfig messages for Tofino, and all Tofino-based platforms can set a size greater to this bound when starting the NOS (e.g. Stratum). I actually exposed this configuration parameter as a command-line argument in Stratum, and it can be leveraged by any deployment mechanism we choose for Stratum.
That might be a misleading way to ask the question in the summary line, but here are some specific examples:
Suppose a client tries to send 17 billion entity updates in a single write request batch. What should a server unable to store that entire message do?
Is there some "minimum batch size" that all server implementations should be able to support?
Similarly for any other message with a 'repeated' field in a client->server message (perhaps we can ignore repeated things for action parameters or table key fields, since those will typically be pretty small for successfully compiled P4 programs, but maybe not, since a client can create erroneous long messages, e.g. with the same field id millions of times).