numaproj / numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs
https://numaflow.numaproj.io
Apache License 2.0
1.01k stars 98 forks source link

Use protobuf for payload and header serde #1768

Closed vigith closed 5 days ago

vigith commented 1 week ago

Summary

We use encoding/binary for serde. We need to standardize on a wire format that is scalable and also supports backward compatibility. encoding/binary encourages using gob or protobuf; I think protobuf has more benefits than gob because it is language agnostic and streaming is not implicit.


Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.