ref: A better commit log format

getsentry / arroyo

A library to build streaming applications that consume from and produce to Kafka.

https://getsentry.github.io/arroyo/

Apache License 2.0

41 stars 7 forks source link

Closed lynnagara closed 1 year ago

lynnagara commented 1 year ago

I really don't want to port this abomination to Rust, let's simplify the commit log format

Two main changes:

uses float instead of weird datetime format we had before
avoids putting important information in headers. This was a hack to avoid changing the message format earlier, but it should be avoided. The timestamp is now in the payload of the message

This is the first of 3 changes, encode method is unchanged, the decode method supports the new format.

lynnagara commented 1 year ago

On second thoughts, gonna move the encode change into a separate PR for deploy safety reasons