Closed ghost closed 4 years ago
Hi @karek314 thanks for the proposal this is great!
We have already a few fixes for the block sync, but far from enough.
So far the implementation is the following:
bottom-line: decentralised block chain should ship the accounts status at some round levels (how many lisks and votes for accounts) to be really efficient imho.
LIP001 - Lisk Improvement Proposal 1 (Improving network synchronisation - decentralised blockchain snapshots)
Topic has been raised in issue #232. Currently syncing new node from beginning is totally unstable, even when snapshots are pretty new it still takes lots of time. Process become very slow, syncing one old block sometimes takes even more time than network produce new block which makes node impossible to catchup with network, sometimes syncing suddenly can speed up for a while to import couple blocks per second, anyway currently syncing is bugged but i know team is working on that i have also saw some new commits in regards to that. Currently lisk team occasionally provide blockchain snapshot stored on lisk.io. Which is great way to remove some unnecessary load from nodes on cpu-level but it's also centralised and relay on one operator.
I think official snapshots should be removed and replaced with decentralised snapshots. Blockchain snapshots is a great idea to improve syncing efficiency. Optionally node operator could allow distribution of blockchain snapshots. Blockchain splitted to packages of - for example 256 or 512 or 5000 blocks, or packs of 128mb with N blocks depending on block size. Each pack can be zipped or compressed with any other maybe more efficient method. It should save some unnecessary cpu load and bandwidth and improve syncing new nodes but it will require more space from node operator so it should be as optional feature and possibly as service in upcoming feature "Standby delegate Market". Each node which is willing to help network sync at the maximum possible level of speed and stability could simply add one flag in config which allow to create and distribute "decentralised snapshots". Each N blocks new package is being created and newly introduced node can simply at efficient speed acquire all packages, uncompress and start rebuilding and verification while another one is downloading from this or another node. But similar solution could be achieved with improvements to network sync. This is proposal to replace official snapshots with decentralised snapshots or possibly improve entire network syncing speed and stability to "snapshot level" and logging while local blockchain height is more than 200 blocks behind network could me improved to more readable format like "Importing blocks 0-256".