filecoin-project / cpp-filecoin

C++17 implementation of Filecoin
https://filecoin.io/
GNU General Public License v3.0
126 stars 36 forks source link

Sync messages after blocks #526

Closed turuslan closed 2 years ago

turuslan commented 2 years ago

Currently sync process (sync_job.cpp) does not start interpreting until it downloads all blocks and messages between head and target. It fetches blocks and related messages in single blocksync request, iteratively from target (newer) to head (older). Sync has index of branches which are already fetched, it assumes that they have all blocks and messages.

Lotus starts by fetching blocks only (without messages it gives more blocks faster) to find chain of parent hashes. Then it lazily fetches messages which are required to interpret next tipset, iteratively from head to target.

Refactoring sync process would allow to start interpreting earlier, and reduce catchup time. Sync index will then contain branches with fetched blocks, but may have no messages. Messages will be requested when required by interpretation (may prefetch more messages if there is time or make parallel requests for multiple heights). That also requires to keep track of connected peers and head they have, because we need peer to request messages from.