near / nearcore

Reference client for NEAR Protocol
https://near.org
GNU General Public License v3.0
2.32k stars 623 forks source link

feat: Mark PartialChunks as non-Cold column #12105

Closed tayfunelmas closed 1 month ago

tayfunelmas commented 1 month ago

This is re-submission of part of #12029, which was reverted in #12085.

In this PR, we only stop saving PartialChunks to Cold DB, BUT we dot yet clear the existing data in Cold DB. The clearing of the data will be implemented in separate PR, since the previous attempt #12029 had issues with large data size being not cleared.

Context

PartialChunk is not needed for replaying the chain history and answering historical queries. It can be re-constructed from other information such as Chunks column. Thus, we remove it from Cold DB. This will save ~9 TB disk space for archival nodes (dashboard).

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 71.52%. Comparing base (8ece1ac) to head (dbc29f8). Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #12105 +/- ## ========================================== - Coverage 71.53% 71.52% -0.01% ========================================== Files 818 818 Lines 164494 164495 +1 Branches 164494 164495 +1 ========================================== - Hits 117667 117658 -9 - Misses 41688 41695 +7 - Partials 5139 5142 +3 ``` | [Flag](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | Coverage Δ | | |---|---|---| | [backward-compatibility](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.17% <0.00%> (ø)` | | | [db-migration](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.17% <0.00%> (ø)` | | | [genesis-check](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.26% <0.00%> (-0.01%)` | :arrow_down: | | [integration-tests](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `38.65% <100.00%> (+<0.01%)` | :arrow_up: | | [linux](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.32% <100.00%> (-0.01%)` | :arrow_down: | | [linux-nightly](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.11% <100.00%> (-0.01%)` | :arrow_down: | | [macos](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `52.59% <0.00%> (-0.36%)` | :arrow_down: | | [pytests](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.53% <0.00%> (-0.01%)` | :arrow_down: | | [sanity-checks](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.33% <0.00%> (-0.01%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `65.24% <0.00%> (-0.02%)` | :arrow_down: | | [upgradability](https://app.codecov.io/gh/near/nearcore/pull/12105/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.21% <0.00%> (-0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.