gchq / sleeper

A cloud-native, serverless, scalable, cheap key-value store
Apache License 2.0
53 stars 11 forks source link

Model separate compaction commit status update #2775

Closed patchwork01 closed 4 days ago

patchwork01 commented 1 week ago

Background

Split from:

Description

We'd like to be able to add a separate status update for when a compaction is committed to the state store, after it has finished in a compaction task.

We will follow up on this to actually store the new events in DynamoDB, and to add them to the status store in a task and the state store committer. We'd like to still be able to support a combined status update for when a compaction is committed synchronously.

Analysis

This should be fairly similar to ingest, see https://github.com/gchq/sleeper/issues/2731.