search

bacalhau-project / bacalhau

Compute over Data framework for public, transparent, and optionally verifiable computation
https://docs.bacalhau.org
Apache License 2.0
649 stars 86 forks source link

Implement protobuf messages for compute node and executor RPC #2700

Open frrist opened 11 months ago

frrist commented 11 months ago

When pluggable executors are enabled, the executors and compute node may communicate over RPC, utilizing protocol buffers for serialization (gRPC). In #2637, we implemented this new behavior, but deliberately opted not to define specific protobuf messages for each RPC request/response type. Currently, we find using JSON bytes more practical, anticipating that many structures passed between the compute node and executor will be further refactored. This approach helps us avoid the need to continuously rewrite and regenerate the protobuf code as we make ongoing changes to the models.

Once the contents of the messages exchanged between the compute node and executor stabilize, we should further refine their protocol definition. This means defining the actual messages they are passing as protobufs with their respective fields, rather than merely wrapping serialized JSON bytes in protobuf containers. It's a step towards making our communication more precise and structured

frrist commented 11 months ago

This issue is currently blocked on #2701