redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.46k stars 579 forks source link

iceberg: value serialization to/from bytes #23481

Closed andrwng closed 2 days ago

andrwng commented 2 days ago

Adds byte serialization for primitive types[1].

Manifests and manifest lists will need to store serialized partition keys, and thus, need to serialize various primitive values as bytes.

The implementation for the most part is lifted from the implementation in serde/rw/scalar.h, which also stores values as little endian.

[1] https://iceberg.apache.org/spec/#binary-single-value-serialization

Backports Required

Release Notes