[changelog] Add message compaction as an opt in feature to changelog consumer - Githubissues

linkedin / venice

Venice, Derived Data Platform for Planet-Scale Workloads.

https://venicedb.org

BSD 2-Clause "Simplified" License

487 stars 84 forks source link

[changelog] Add message compaction as an opt in feature to changelog consumer #1230

Closed ZacAttack closed 25 minutes ago

ZacAttack commented 3 weeks ago

[changelog] Add message compaction as an opt in feature to changelog consumer

This introduces a configuration which will compact down data returned from poll to only contain the latest records for a given key. This also maintains the order of results returned.

Resolves #XXX

How was this PR tested?

Does this PR introduce any user-facing changes?

[ ] No. You can skip the rest of this section.
[ ] Yes. Make sure to explain your proposed changes and call out the behavior change.