gchq / sleeper

A cloud-native, serverless, scalable, cheap key-value store
Apache License 2.0
53 stars 11 forks source link

Distinguish between finished and committed in compaction jobs report #2768

Closed patchwork01 closed 2 days ago

patchwork01 commented 1 week ago

Background

Split from:

Description

We'd like to add information to the compaction jobs report about when files have been committed for the job.

This should include clarifying when a job has finished, but has not been committed to the state store. A job should only count as finished after all files have been committed.

Analysis

This should be similar to ingest, as in https://github.com/gchq/sleeper/issues/2726.