basho / riak

Riak is a decentralized datastore from Basho Technologies.
http://docs.basho.com
Apache License 2.0
3.92k stars 534 forks source link

Failed to release #1017

Closed kpy3 closed 3 years ago

kpy3 commented 4 years ago

Hi,

just discovered a lot of messages like

2020-04-10T15:16:39.792  log_level=info log_ref=P0039 db_id=28 pid=<0.1559.0> Failed to release pid=r2d_fail leaving SnapshotCount=2 and MinSQN=21715
2020-04-10T15:16:39.830  log_level=info log_ref=P0039 db_id=28 pid=<0.1559.0> Failed to release pid=r2d_fail leaving SnapshotCount=2 and MinSQN=21715
2020-04-10T15:16:39.830  log_level=info log_ref=P0039 db_id=28 pid=<0.1559.0> Failed to release pid=r2d_fail leaving SnapshotCount=2 and MinSQN=21715
2020-04-10T15:16:39.839  log_level=info log_ref=P0039 db_id=28 pid=<0.1559.0> Failed to release pid=r2d_fail leaving SnapshotCount=2 and MinSQN=21715
2020-04-10T15:16:39.841  log_level=info log_ref=P0039 db_id=28 pid=<0.1559.0> Failed to release pid=r2d_fail leaving SnapshotCount=2 and MinSQN=21715

What does this message actually mean?

martinsumner commented 4 years ago

It shouldn't be a cause for immediate concern. Folds against leveled take a read-only snapshot of the ledger to give a consistent view of the state of the store at the time the fold commenced.

When the fold completes, they should tidy up and de-register themselves (which allows for ledger file deletes blocked due to the snapshot to complete). If a fold fails to de-register within a timeout it will be killed to free-up the store again.

When a file in the ledger is ready to delete (this will happen quite often) it checks there are no outstanding snapshots which require it to be present. If not, it can erase itself, otherwise it runs a check to see if any of the snapshots which were blocking the delete have timed out - and this check prompts the log P0036 with pid=r2d_fail.

This can be a bit noisy in the logs, as often many files will be ready to delete at the same time (after a big merge). If there is a snapshot running, then they will all prompt the log.