HathorNetwork / hathor-core

Hathor core is the official and reference client for operating a full node in Hathor Network.
https://hathor.network
Apache License 2.0
83 stars 26 forks source link

refactor(benchmark): change benchmark CI to download data from logs in AWS #1133

Closed glevco closed 1 day ago

glevco commented 3 weeks ago

Motivation

Currently, CI benchmark directly connects to testnet to download benchmark data (it performs a sync all the way up to 20k blocks). This PR changes it to use the latest load-from-logs CLI tool update, making the CI run faster and more reliably. Then, we'll also be able to introduce new CI benchmarks in other PRs, including for example syncing a "dummy" network with more transactions than blocks.

Acceptance Criteria

Checklist

github-actions[bot] commented 3 weeks ago

🐰 Bencher Report

Branchrefactor/bench-data
Testbedubuntu-22.04
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Lower Boundary
nanoseconds (ns)
(Limit %)
Upper Boundary
nanoseconds (ns)
(Limit %)
sync-v2 (up to 20000 blocks)📈 view plot
🚷 view threshold
101,471,694,713.76
(-0.55%)
91,826,613,182.35
(90.49%)
112,232,527,222.88
(90.41%)
🐰 View full continuous benchmarking report in Bencher
codecov[bot] commented 3 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.36%. Comparing base (7f53032) to head (63d11c1). Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #1133 +/- ## ========================================== + Coverage 84.33% 84.36% +0.02% ========================================== Files 317 317 Lines 24297 24297 Branches 3697 3697 ========================================== + Hits 20491 20498 +7 + Misses 3083 3078 -5 + Partials 723 721 -2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

glevco commented 2 weeks ago

@jansegre

Should we call it something else other than "sync_v2" now? Using load-from-logs shouldn't trigger any sync code, right? Maybe "load_sample". Other than that it looks good to me.

The benchmark is still the same, it uses Sync-v2. I only changed the way the server node boots up (from syncing with the network vs from logs). Then, a second node connects to the server node and the measurement is made on the sync of that second node.