search

paradigmxyz / reth

Modular, contributor-friendly and blazing-fast implementation of the Ethereum protocol, in Rust
https://reth.rs/
Apache License 2.0
3.52k stars 910 forks source link

move `ChunkedFileReader` type into `FileClient` type #7650

Open emhane opened 2 months ago

emhane commented 2 months ago

Describe the feature

Incorporate the ChunkedFileReader into the FileClient, as opposed to ChunkedFileReader::next_chunk returning a new FileClient. Instead, a new chunk is read upon call to <FileClient as HeadersClient>::get_headers_with_priority, if the FileClient has no more headers in its map.

This requires modifying the ImportCommand. A new arg is needed --tip <block-hash-sync-target>, which must be passed to the FileClient. The pipeline can then be built once.

Additional context

As a side effect, this will enable ETL wrt to the whole file on chunked reads.

No response

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 21 days with no activity.

emhane commented 1 month ago

headers downloader gets tip to genesis, makes this more complex, since requires reading the file in reverse. right @joshieDo ?