search

near / nearcore

Reference client for NEAR Protocol
https://near.org
GNU General Public License v3.0
2.32k stars 623 forks source link

feat: Update compact-database command to enable compacting cold storage #12094

Open tayfunelmas opened 1 month ago

tayfunelmas commented 1 month ago

Current neard database compact-database command runs compaction on Hot db by default.

We add an option to run compaction on cold db. We will also use this to run some experiments on cleaning up data in cold storage followed-up with compaction.

The command continues to run on Hot DB by default and cold storage is only enabled by adding a -t cold flag.

Also change utils to use DBOpener to fix a bug about not opening the right db path (previously home directory path was not joined with the db directory name).

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 23.07692% with 20 lines in your changes missing coverage. Please review.

Project coverage is 71.82%. Comparing base (3d46ee4) to head (5c5d00a). Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
tools/database/src/utils.rs 0.00% 16 Missing :warning:
tools/database/src/compact.rs 0.00% 4 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #12094 +/- ## ======================================= Coverage 71.82% 71.82% ======================================= Files 827 827 Lines 166626 166646 +20 Branches 166626 166646 +20 ======================================= + Hits 119681 119696 +15 - Misses 41728 41734 +6 + Partials 5217 5216 -1 ``` | [Flag](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | Coverage Δ | | |---|---|---| | [backward-compatibility](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.17% <0.00%> (-0.01%)` | :arrow_down: | | [db-migration](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.17% <0.00%> (-0.01%)` | :arrow_down: | | [genesis-check](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.25% <0.00%> (-0.01%)` | :arrow_down: | | [integration-tests](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `38.77% <23.07%> (+0.01%)` | :arrow_up: | | [linux](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.51% <23.07%> (-0.01%)` | :arrow_down: | | [linux-nightly](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.40% <23.07%> (+<0.01%)` | :arrow_up: | | [macos](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `54.53% <23.07%> (-0.07%)` | :arrow_down: | | [pytests](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.57% <0.00%> (-0.01%)` | :arrow_down: | | [sanity-checks](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.37% <0.00%> (-0.01%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `65.63% <23.07%> (-0.01%)` | :arrow_down: | | [upgradability](https://app.codecov.io/gh/near/nearcore/pull/12094/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.21% <0.00%> (-0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.