maticnetwork / heimdall

Validator node for Polygon PoS
https://polygon.technology/
GNU General Public License v3.0
272 stars 181 forks source link

bor: increase max limit for list span api #1180

Closed shohamc1 closed 2 months ago

shohamc1 commented 2 months ago

Description

The max limit of spans/list api is set to 20 which is very small. Currently each span is around 6 KB, we can fit ~160 in a 1 MB response size. This PR sets the limit to 150 to account for future span size growth.

Context: Erigon would like to efficiently fetch spans from heimdall for an improved initial sync mechanism. We would like to batch fetch spans cut down the number of requests sent to the heimdall service.

Changes

Checklist

Cross repository changes

Testing

Manual tests

➜  ~ curl 'http://localhost:1317/bor/span/list?page=1&limit=10' | jq | grep "span_id" | wc -l
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  132k    0  132k    0     0   173k      0 --:--:-- --:--:-- --:--:--  173k
      10
➜  ~ curl 'http://localhost:1317/bor/span/list?page=1&limit=150' | jq | grep "span_id" | wc -l
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2830k    0 2830k    0     0  12.7M      0 --:--:-- --:--:-- --:--:-- 12.7M
     150
➜  ~ curl 'http://localhost:1317/bor/span/list?page=1&limit=200' | jq | grep "span_id" | wc -l
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2830k    0 2830k    0     0  18.6M      0 --:--:-- --:--:-- --:--:-- 18.6M
     150
codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 76.36%. Comparing base (ae5a160) to head (65acc04). Report is 34 commits behind head on develop.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #1180 +/- ## =========================================== - Coverage 76.40% 76.36% -0.04% =========================================== Files 53 53 Lines 5929 5932 +3 =========================================== Hits 4530 4530 - Misses 1139 1141 +2 - Partials 260 261 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.