gchq / sleeper

A cloud-native, serverless, scalable, cheap key-value store
Apache License 2.0
53 stars 11 forks source link

Store separate compaction commit status update in DynamoDB #2766

Closed patchwork01 closed 3 days ago

patchwork01 commented 1 week ago

Background

Split from:

Description

We'd like to be able to store a separate status update for when a compaction is committed to the state store, after it has finished in a compaction task.

We will follow up on this to actually report the events separately. We'd like to still be able to support a combined status update for when a compaction is committed synchronously.

Analysis

This should be fairly similar to ingest, see https://github.com/gchq/sleeper/issues/2723.

We'll also need to store the job run ID.