Add (optional) RPC for event-based messages e. g. osi_control and osi_graphics

[globberwops]

Discussed in https://github.com/orgs/OpenSimulationInterface/discussions/675

^{Originally posted by **globberwops** October 19, 2022} Dear all, `osi_control` and `osi_graphics` (and possibly other packages?) will introduce *event-based messages* into OSI (see project proposal P2022-01 Performance & Packaging WP5 "control messages"). In parallel, there is the ongoing [discussion](https://github.com/OpenSimulationInterface/open-simulation-interface/discussions/625) about switching from the current protobuf message description to flatbuffers (see project proposal P2022-01 Performance & Packaging WP1). *Event-based messages* would benefit greatly from an RPC (remote procedure call) implementation. [gRPC](https://grpc.io/) for example, extends both protobuf and flatbuffers by RPC. The *event-based messages* could therefore be defined as gRPC services ([example](https://grpc.io/docs/languages/cpp/basics/)). The generation of gRPC code could then be made optional in the CMake build, so that users can choose to use plain protobuf/flatbuffers, or additionally the RPC. On a side note, the [original issue](https://github.com/OpenSimulationInterface/open-simulation-interface/issues/56) also mentions [CapnProto](https://capnproto.org/), which would add RPC as well. Regards, Martin Martin Stump on behalf of MBition GmbH, [Provider Information](https://github.com/mercedes-benz/foss/blob/master/PROVIDER_INFORMATION.md)

[globberwops]

As discussed in the WP5 meeting on 15 Dec 2022 we will proceed as follows:

1. Create a repo for the new packaging layer using gRPC e.g. osi-grpc-packaging
2. Add the service definitions in the new repo, closely coordinating with the message definitions group
3. Add a reference implementation of the "application interface" in the new repo

[tbleher]

I think an RPC service would help a lot in certain situations. I'm a bit worried about gRPC however, since it seems really huge (just look at all its dependencies: https://github.com/grpc/grpc/blob/master/.gitmodules).

Is there implementation experience using gRPC? Are there smaller RPC services that could be used (e.g. something not requiring a complete HTTP/2 + SSL implementation)?

[tbleher]

https://github.com/protocolbuffers/protobuf/blob/main/docs/third_party.md#rpc-implementations lists some RPC systems with support for ProtoBuf, though none (at first glance) seems to support as many languages as gRPC.

[globberwops]

@tbleher gRPC has the widest adoption by far of any of the proto-based RPC implementations. While it surely is not the lightest library by any means, it is widely available through all major package managers (apt, conan, vcpkg, ...), so availability should not be an issue in my opinion.

[engelben]

@pmai @globberwops we can set up a new repo but I need a bit more insight into this before we can formally approve this for release in an ASAM OSI standard. We need to make sure that the decision to include the gRPC dependency (even if optional) in OSI itself is a full project decision - I see a final integration as dependent on a project vote/approval followed by TSC approval. This should be based on a clear overview of the motivation, the alternatives and a short discussion to answer any questions. This should also clarify any licensing aspects. Secondly is the reference implementation - if we are doing this as part of the ASAM project then it needs to go through a similar procedure as above.

I can create the repo so that the work is not blocked but I would request you to address the above in parallel. The next ASAM TSC is on 18.04, with the submission deadline for presentation content on the 31.03. Let me know if you want me to plan you in for this or if you would like to target a later TSC. @pmai once I get the OK from you I will get the ball rolling. Another question: Who will integrate the new repo into the pipeline?

[pmai]

@engelben To clarify: The repo is for initial development of such a packaging layer, which would, if adopted, be another optional packaging layer. Due to logistical reasons, it is preferred to use a common repo for initial development and drafting, prior to an official decision by the project and/or TSC on whether to include it or not. The Repo description can clearly spell out this situation. So to reiterate I am in favor of the repo creation.

[florianwallerer]

Repo is created by @engelben with a MPL 2.0 license. see: https://github.com/OpenSimulationInterface/osi-grpc-packaging

[globberwops]

@engelben Thanks for creating the repo!

@ThomasNaderBMW @pmai Here's the initial design proposal for osi_application_control.

There are 2 scenarios:

1. Application Control: The client (e.g. the simulation host) calls functions on the server (e.g. a graphics engine) to init/start/pause/stop/... the engine.
2. Symmetric Delta: The client (e.g. the graphics engine) calls functions on the server (e.g. the simulation host) to retrieve updates.

The message osi_application_control is designed to be usable with and without gRPC. The file osi_application_control.proto should go into the main OSI repo once completed, and replace the current osi_control proposal.

By putting all the meta info in a separate submessage, we can do fine-grained unary RPC calls, as shown here.

The unit tests show how the RPC would be called by a client.

Looking forward to our discussion.

Cheers, Martin

Edit: I believe I mixed up the server/client roles for the symmetric delta. I'll fix it until the next meeting.

[globberwops]

[globberwops]

ASAM OSI | Subgroup Performance Meeting | 4.0.0 Development | 2023-10-19

• Apply the same approach to TrafficParticipant (re. message structuring)
• Client/server roles: standardize setup phase?
• Scope: All of OSI or only control-type messages?
  • Start with control-type messages
• Document design considerations
• Design goal: reusability

Review of the initial design proposal

Focus on the how instead of the what at first:

• Structure of the ApplicationControl message
• Granularity of the RPC definitions
• Usability of the client/server stubs

[globberwops]

ASAM OSI | Subgroup Performance Meeting | 4.0.0 Development | 2023-11-30

• Hubert Cao, Thomas Hempen, Jason Zhang, Martin Stump will be actively working on the osi-gprc-packaging layer
• Request-response-type (or event-based) messages:
  • TrafficCommand -> Commands to models
  • TrafficCommandUpdate
  • ApplicationControl -> Commands to external applications (e.g. graphics engine)
• Structure of the event-based messages should be based on TrafficCommand

Next steps:

1. Redesign ApplicationControl based on TrafficCommand
2. Organize a focus meeting with active parties to define the initial services