search

real-logic / simple-binary-encoding

Simple Binary Encoding (SBE) - High Performance Message Codec
Apache License 2.0
3.12k stars 523 forks source link

[Go] no zero-copy? #765

Open asido opened 4 years ago

asido commented 4 years ago

I generated Go code and see there is no Wrap() method. Instead, there is Encode(io.Reader), which fetches and assigns all values to public fields in the model. Is there no zero-copy alternative in Go, that allows to read the data in-place?

neomantra commented 3 years ago

This is correct. The current Go implementation takes raw SBE message and creates a fully-formed Golang struct from it and thus every field is marshaled.

The reasoning behind this is described here: https://github.com/real-logic/simple-binary-encoding/wiki/Golang-User-Guide#golang-idioms

This design prioritizes easy-to-use and hard-to-misuse over performance. It should be possible to make a zero-copy / flyweight generator if someone were up to the task.

matthewdowney commented 1 year ago

@ethanf curious if your impl ended up looking promising?

ethanf commented 1 year ago

@ethanf curious if your impl ended up looking promising?

I've updated https://github.com/ethanf/simple-binary-encoding/tree/go-flyweight with the latest code. It's code complete and mostly works, but there's still work to do updating the tests unfortunately.

ethanf commented 1 year ago

https://github.com/ethanf/simple-binary-encoding/tree/go-flyweight has passing tests now, does anyone have a preference for how to switch between the flyweight and struct versions of the generated code? Is a system property that switches between the generators acceptable?

neomantra commented 1 year ago

Thanks for your contribution! I only started looking at it and will take it for a spin in a few weeks.

I do think that using system properties like that is consistent with other SBE / Aeron tool options.