FuelLabs / fuel-core

Rust full node implementation of the Fuel v2 protocol.
Other
57.96k stars 2.78k forks source link

`fuel-core-sync` should cache the result of responses instead of throwing them away #2357

Open xgreenx opened 1 month ago

xgreenx commented 1 month ago

Overview

The fuel-core-sync service has a single stream of events where processes result sequentially. In several parts of the stream, we request data from another peer in parallel to speed up synchronization.

If any requests fails, it shuts down the stream and cleans everything. The problem is that we throw away results that maybe be useful when we start this stream next time.

Definition of done

The service doesn't fetch data that already was known and valid and keeps it instead.

AurelienFT commented 1 month ago

When you are talking about any requests fails. I don't see a lot of error returned etc except here : https://github.com/FuelLabs/fuel-core/blob/da1f69fa1ef16212473fda4b46d7f32cbb46f255/crates/services/sync/src/import.rs#L321 and here : https://github.com/FuelLabs/fuel-core/blob/da1f69fa1ef16212473fda4b46d7f32cbb46f255/crates/services/sync/src/import.rs#L250

Is that what you meant by "request fails" ?

Also what do you means by data that were valid ? Consensus valid + Execution valid ?

xgreenx commented 1 month ago

If any of the requests fails, we return None and into_scan_none will stop the stream, even if we have next 999 blocks by the first block failed.

image
AurelienFT commented 1 month ago

@xgreenx If you have some time this evening I would love a confirmation that this test test the future expected behavior : https://github.com/FuelLabs/fuel-core/blob/704a5837ec4677d40ee4f68d427ef15c9a8ccdda/crates/services/sync/src/import/tests.rs#L251