API Question: Is it possible to broadcast without memcpy?

[vasigavr1]

Hello Anuj,

Is it possible to enqueue the same request to multiple sessions using the same buffer space (i.e. "broadcast" by performing multiple unicasts)?

Allow me to elaborate. From what i can tell, in order to perform a "broadcast", we must create multiple MsgBuffers -- one per each distinct RPC, because once a MsgBuffer is passed to eRPC (through enqueue_request()), it is no longer ours to pass it to enqueue_request() again (not until a response is received anyway, which would be too late).

Is it possible to enqueue multiple MsgBuffers, whose .buf point to the same location? If not that leaves only one solution to broadcasting an RPC: creating one MsgBuffer per message and memcpying the .buf to all MsgBuffers. (But this seems unnecessary.)

Am I missing something?

Thank you very much, vasilis

[anujkaliaiitd]

Thanks for bringing this up. eRPC's current design appears incompatible with this copyless broadcast optimization. The issue is that eRPC co-locates packet headers with application data (Figure 2 in paper). For broadcast RPCs, we wish to have identical data but different packet headers (e.g., the request number, destination session number, L2--L4 headers in Ethernet mode), but in eRPC, data and headers are tied into MsgBufs.

Reorganizing MsgBuffers to decouple packet headers and application data buffers seems like a promising solution. We had an internal project at CMU that reorganized the header layout in a different way (while still co-locating headers and data), so doing so isn't too complex.

I've tried to be consistent in using get_pkthdr_0() and get_pkthdr_n() to indirectly retrieve packet headers from MsgBufs, instead of direct pointer math on msgbuf.buf. This indirection allows locating all the code points where headers are used, as well as overriding the association between msgbuf data and headers.

[vasigavr1]

Thank you very much!