Cannot remove data object if final replica is stale

[korydraughn]

• [x] main
• [x] 4-3-stable

Bug Report

Tested against potential 4.3.2 build of server and plugin.

Saw the following when the REP was enabled.

$ ils -l
/tempZone/home/kory:
  kory              0 demoResc            0 2024-04-01.10:52 X foo
$ irm -f foo
remote addresses: [IP REDACTED] ERROR: rmUtil: rm error for /tempZone/home/kory/foo, status = -1205000 status = -1205000 RE_RUNTIME_ERROR
Level 0: no good replica found: SYS_NO_GOOD_REPLICA

The log contained the following message.

"log_message": "no good replica found: SYS_NO_GOOD_REPLICA",

[trel]

And this works fine without logical quotas enabled?

[korydraughn]

Yes.

[korydraughn]

The issue occurs here because that filesystem function filters out stale replicas. https://github.com/irods/irods_rule_engine_plugin_logical_quotas/blob/b285ba0ac1e5c422621b067ea63b88a5e9d47b3f/src/handler.cpp#L1199

See its implementation for specifics. https://github.com/irods/irods/blob/cd6b746802c6a5eaffb4f40a53cbe70d090c5f72/lib/filesystem/src/filesystem.cpp#L744-L763

The REP is designed to track the data size of data objects with at least one good replica. What should the REP do if there aren't any good replicas? Below are some ideas.

• (a) Leave the data size value as is
• (b) Subtract the size of the replica having the latest mtime

Option (a) seems like the best answer for the following reasons.

• Replicas can become stale for multiple reasons, sometimes for unknown reasons
• Replicas can be trimmed - what happens if the most recent replica is trimmed?
• The REP gives admins a rule which can recalculate totals, which should be used to resolve accuracy/drift issues

Tracking the number of data objects is not affected by this.

[trel]

Also, (a) makes sense because logical quotas is already 'ignoring' any object without a good replica.

Agreed, we should have a 'recalculate' rule... to manually correct any 'drift'.

[korydraughn]

Related to #94.