HathorNetwork / hathor-core

Hathor core is the official and reference client for operating a full node in Hathor Network.
https://hathor.network
Apache License 2.0
83 stars 26 forks source link

feat: improve load-from-logs CLI command #1132

Closed glevco closed 4 weeks ago

glevco commented 4 weeks ago

Motivation

Currently, the load-from-logs CLI command reads a raw logs file as outputted by a full node. This PR changes it so instead you have to process the log file with the parse-logs command, only then to use it with load-from-logs. This makes the log files much smaller, as the parser strips every information but the vertex bytes.

Also, previously that command only called on_new_tx in a loop, but didn't actually run the reactor or the full node itself. This PR also changes this.

The new intended workflow would be something like this:

# Run a full node and save its logs to a file
$ poetry run hathor-cli run_node --testnet --memory-storage --json-logs --log-vertex-bytes &> json_logs
# After stopping the full node, parse the logs file
$ poetry run hathor-cli parse-logs --output-file parsed_logs --json-logs-file json_logs
# Run load-from-logs to create a full node that receives each vertex from the log file
$ poetry run hathor-cli load-from-logs --testnet --memory-storage --x-localhost-only --log-dump parsed_logs

Acceptance Criteria

Checklist

github-actions[bot] commented 4 weeks ago

🐰Bencher

ReportThu, September 5, 2024 at 15:22:31 UTC
Projecthathor-core
Branchfeat/improve-load-from-logs
Testbedubuntu-22.04
Click to view all benchmark results
BenchmarkLatencyLatency Results
nanoseconds (ns) | (Δ%)
Latency Lower Boundary
nanoseconds (ns) | (%)
Latency Upper Boundary
nanoseconds (ns) | (%)
sync-v2 (up to 20000 blocks)✅ (view plot)101,409,921,726.48 (-0.77%)91,981,574,820.96 (90.70%)112,421,924,781.17 (90.20%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help
codecov[bot] commented 4 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.41%. Comparing base (50ae3b2) to head (0703970). Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #1132 +/- ## ========================================== + Coverage 84.39% 84.41% +0.01% ========================================== Files 317 317 Lines 24240 24240 Branches 3684 3684 ========================================== + Hits 20458 20462 +4 + Misses 3069 3065 -4 Partials 713 713 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.