near / nearcore

Reference client for NEAR Protocol
https://near.org
GNU General Public License v3.0
2.32k stars 623 forks source link

refactor(network): SnapshotHostsCache #12103

Closed saketh-are closed 1 month ago

saketh-are commented 1 month ago

This module is dead code in master but I've started using it in #12095. While testing end-to-end I noticed some unexpected behavior and started digging into the implementation. While doing so I ended up refactoring it extensively and simplifying it somewhat. Ultimately I don't think there was actually any bug.

One meaningful change I've made is to include the shard id in the priority score. This ensures that in the specific situation that different shards have the same/similar host set and the number of parts is smaller than the number of hosts, the load will still be well distributed. This is admittedly not a concern in mainnet (where the number of parts should be large) but can easily occur in localnet.

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 90.62500% with 9 lines in your changes missing coverage. Please review.

Project coverage is 71.53%. Comparing base (773aae7) to head (642e9d7). Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
chain/network/src/snapshot_hosts/mod.rs 89.41% 8 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #12103 +/- ## ========================================== - Coverage 71.59% 71.53% -0.06% ========================================== Files 818 818 Lines 164532 164494 -38 Branches 164532 164494 -38 ========================================== - Hits 117792 117667 -125 - Misses 41597 41687 +90 + Partials 5143 5140 -3 ``` | [Flag](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | Coverage Δ | | |---|---|---| | [backward-compatibility](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.17% <0.00%> (+<0.01%)` | :arrow_up: | | [db-migration](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.17% <0.00%> (+<0.01%)` | :arrow_up: | | [genesis-check](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.26% <0.00%> (+<0.01%)` | :arrow_up: | | [integration-tests](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `38.68% <10.41%> (-0.01%)` | :arrow_down: | | [linux](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.32% <90.62%> (+0.12%)` | :arrow_up: | | [linux-nightly](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.12% <90.62%> (-0.01%)` | :arrow_down: | | [macos](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `52.59% <7.22%> (-0.86%)` | :arrow_down: | | [pytests](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.53% <0.00%> (+<0.01%)` | :arrow_up: | | [sanity-checks](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.33% <0.00%> (+<0.01%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `65.25% <90.62%> (-0.13%)` | :arrow_down: | | [upgradability](https://app.codecov.io/gh/near/nearcore/pull/12103/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.21% <0.00%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.