gchq / sleeper

A cloud-native, serverless, scalable, cheap key-value store
Apache License 2.0
53 stars 11 forks source link

Add compaction finished status update in task, commit update in state store lambda #2767

Closed patchwork01 closed 2 days ago

patchwork01 commented 1 week ago

Background

Split from:

Follows on from:

Currently only the compaction finished status update is added to the status store, in either the compaction task or the state store commit lambda, depending on whether the job is committed synchronously or asynchronously.

Description

We'd like to add the compaction finished status update immediately in the compaction task when a job finishes, even when it will be committed asynchronously.

We'd like to change the state store commit lambda to add a commit update instead of a finished update for a compaction job.

These need to be done together to ensure that a job will have a consistent history of status updates.

Analysis

CompactionJobCommitterOrSendToLambda will need to change significantly.