SEMICeu / LinkedDataEventStreams

The Linked Data Event Streams specification
https://tree.linkeddatafragments.org/linked-data-event-streams/
23 stars 9 forks source link

Latest state + time retention #36

Open sandervd opened 1 year ago

sandervd commented 1 year ago

The idea of this retention policy is that it combines the benefits of snapshots with retention. This way, if replication of state is the concern of the client, a log that is as small as possible can be offered. This saves resources for both publisher and subscriber.

The idea of the algorithm is explained here: https://docs.confluent.io/kafka/design/log_compaction.html