Resolves #2787: Index scrubber could enforce early transaction commit

alecgrieser commented 2 weeks ago

The main substantive change here is that the OnlineIndexScrubber.Builder now allows the user to set a time limit and a transaction time limit. This is then honored by logic in IndexinbBase: https://github.com/FoundationDB/fdb-record-layer/blob/695c4053527faf59bcb604abd3fd7bf57fddb11f/fdb-record-layer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java#L935-L944

This should result in operations committing early rather than hitting transaction_too_old exceptions.

This also refactors the way that configuration is specified for the online indexer and online index scrubber so that they share a base-builder implementation. That base builder contains the de-duplicated code. In theory, we could have done this by updating the OnlineIndexScrubber.Builder with two new methods setting the time limits, but this seemed like a good change to make to avoid having as much config drift in the future.

This resolves #2787.

foundationdb-ci commented 2 weeks ago

Result of fdb-record-layer-pr on Linux CentOS 7

Commit ID: d928118e304a2aa796c4f8302746b9165b0804b4
Duration 0:48:58
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci commented 2 weeks ago

Result of fdb-record-layer-pr on Linux CentOS 7

Commit ID: 34c1f70c19aa6c8fd3e83e41f1a5b2bec9d84001
Duration 0:51:27
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci commented 2 weeks ago

Result of fdb-record-layer-pr on Linux CentOS 7

Commit ID: f279fb645b63521119eb8ace0ee7e91307a01477
Duration 1:00:24
Result: :x: FAILED
Error: Build has timed out.
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci commented 2 weeks ago

Result of fdb-record-layer-pr on Linux CentOS 7

Commit ID: 92a5311d53eeb1465b00d77fef1135511fc58a25
Duration 0:59:12
Result: :white_check_mark: SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

FoundationDB / fdb-record-layer