fix: 15036: IndexOutOfBoundsException when flushing an empty virtual map to MerkleDb

[artemananiev]

Fix summary:

• A new field is introduced to AbstractLongList and its subclasses: max valid index. Previously there was only a field for min valid index, and list size was used as max valid index
• Both min and max valid indices can be -1, which indicates that the long list is empty
• Before a long list can be used, updateValidKeyRange() method must be called. All updates to the list outside the current range will result in an exception
• startWriting(min, max) method in MIDKVS is split into two: updateValidKeyRange(min, max) and startWriting(). This is needed, because sometimes there is nothing to write, but the store index still needs to be updated. Previously the index was updated directly in startWriting()

Testing:

• A new unit test is provided. It failed before the fix, and passes with the fix
• Existing unit tests are updated to call updateValidKeyRange() when needed

Fixes: https://github.com/hashgraph/hedera-services/issues/15036 Signed-off-by: Artem Ananev artem.ananev@swirldslabs.com

[codacy-production[bot]]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
Report missing for de34f1ca79a8dbc4faa88eb0321e4048714f2dbb[^1]	:white_check_mark: 93.88%

Coverage variation details

| | Coverable lines | Covered lines | Coverage | | ------------- | ------------- | ------------- | ------------- | | Common ancestor commit (de34f1ca79a8dbc4faa88eb0321e4048714f2dbb) | Report Missing | Report Missing | Report Missing | | | Head commit (6fe2c3f2bee780b5c0b0333d27460be5b8373512) | 110866 | 69277 | 62.49% | **Coverage variation** is the difference between the coverage for the head and common ancestor commits of the pull request branch: ` - `

Diff coverage details

| | Coverable lines | Covered lines | Diff coverage | | ------------- | ------------- | ------------- | ------------- | | Pull request (#15097) | 49 | 46 | **93.88%** | **Diff coverage** is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: `/ * 100%`

See your quality gate settings    Change summary preferences

_{Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more} [^1]: Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 83.67347% with 8 lines in your changes missing coverage. Please review.

Project coverage is 59.16%. Comparing base (964661d) to head (6fe2c3f). Report is 38 commits behind head on develop.

Files	Patch %	Lines
...swirlds/merkledb/collections/AbstractLongList.java	70.58%	2 Missing and 3 partials :warning:
.../java/com/swirlds/merkledb/MerkleDbDataSource.java	84.21%	1 Missing and 2 partials :warning:

Additional details and impacted files

[](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph) ```diff @@ Coverage Diff @@ ## develop #15097 +/- ## ============================================= + Coverage 59.10% 59.16% +0.05% - Complexity 22549 22580 +31 ============================================= Files 2902 2897 -5 Lines 111299 111325 +26 Branches 11424 11440 +16 ============================================= + Hits 65783 65861 +78 + Misses 41827 41767 -60 - Partials 3689 3697 +8 ``` | [Files](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph) | Coverage Δ | | |---|---|---| | [.../swirlds/merkledb/collections/LongListOffHeap.java](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&filepath=platform-sdk%2Fswirlds-jasperdb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fswirlds%2Fmerkledb%2Fcollections%2FLongListOffHeap.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph#diff-cGxhdGZvcm0tc2RrL3N3aXJsZHMtamFzcGVyZGIvc3JjL21haW4vamF2YS9jb20vc3dpcmxkcy9tZXJrbGVkYi9jb2xsZWN0aW9ucy9Mb25nTGlzdE9mZkhlYXAuamF2YQ==) | `91.95% <100.00%> (+0.18%)` | :arrow_up: | | [...s/merkledb/files/MemoryIndexDiskKeyValueStore.java](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&filepath=platform-sdk%2Fswirlds-jasperdb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fswirlds%2Fmerkledb%2Ffiles%2FMemoryIndexDiskKeyValueStore.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph#diff-cGxhdGZvcm0tc2RrL3N3aXJsZHMtamFzcGVyZGIvc3JjL21haW4vamF2YS9jb20vc3dpcmxkcy9tZXJrbGVkYi9maWxlcy9NZW1vcnlJbmRleERpc2tLZXlWYWx1ZVN0b3JlLmphdmE=) | `93.18% <100.00%> (-0.30%)` | :arrow_down: | | [...wirlds/merkledb/files/hashmap/HalfDiskHashMap.java](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&filepath=platform-sdk%2Fswirlds-jasperdb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fswirlds%2Fmerkledb%2Ffiles%2Fhashmap%2FHalfDiskHashMap.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph#diff-cGxhdGZvcm0tc2RrL3N3aXJsZHMtamFzcGVyZGIvc3JjL21haW4vamF2YS9jb20vc3dpcmxkcy9tZXJrbGVkYi9maWxlcy9oYXNobWFwL0hhbGZEaXNrSGFzaE1hcC5qYXZh) | `79.03% <100.00%> (ø)` | | | [...irlds/virtualmap/datasource/VirtualDataSource.java](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&filepath=platform-sdk%2Fswirlds-virtualmap%2Fsrc%2Fmain%2Fjava%2Fcom%2Fswirlds%2Fvirtualmap%2Fdatasource%2FVirtualDataSource.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph#diff-cGxhdGZvcm0tc2RrL3N3aXJsZHMtdmlydHVhbG1hcC9zcmMvbWFpbi9qYXZhL2NvbS9zd2lybGRzL3ZpcnR1YWxtYXAvZGF0YXNvdXJjZS9WaXJ0dWFsRGF0YVNvdXJjZS5qYXZh) | `28.57% <ø> (ø)` | | | [.../java/com/swirlds/merkledb/MerkleDbDataSource.java](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&filepath=platform-sdk%2Fswirlds-jasperdb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fswirlds%2Fmerkledb%2FMerkleDbDataSource.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph#diff-cGxhdGZvcm0tc2RrL3N3aXJsZHMtamFzcGVyZGIvc3JjL21haW4vamF2YS9jb20vc3dpcmxkcy9tZXJrbGVkYi9NZXJrbGVEYkRhdGFTb3VyY2UuamF2YQ==) | `82.66% <84.21%> (+1.61%)` | :arrow_up: | | [...swirlds/merkledb/collections/AbstractLongList.java](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&filepath=platform-sdk%2Fswirlds-jasperdb%2Fsrc%2Fmain%2Fjava%2Fcom%2Fswirlds%2Fmerkledb%2Fcollections%2FAbstractLongList.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph#diff-cGxhdGZvcm0tc2RrL3N3aXJsZHMtamFzcGVyZGIvc3JjL21haW4vamF2YS9jb20vc3dpcmxkcy9tZXJrbGVkYi9jb2xsZWN0aW9ucy9BYnN0cmFjdExvbmdMaXN0LmphdmE=) | `92.23% <70.58%> (-2.16%)` | :arrow_down: | ... and [50 files with indirect coverage changes](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph) [](https://app.codecov.io/gh/hashgraph/hedera-services/pull/15097?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=hashgraph)