Fantom-foundation / Sonic

go-opera fork for Carmen and Tosca integration
GNU Lesser General Public License v3.0
19 stars 18 forks source link

Optimize peer selection for initial events download #154

Closed uprendis closed 1 week ago

uprendis commented 3 months ago

For each first attempt in an epoch, pick the previous successful peer in *streamleecher with 98% chance

In mainnet, this optimization increases speed of syncing roughly ~1.5x due to the presence of bots. It cannot be measured accurately due varying number of bots

We may want to replace it with a more complex heuristic in future, such that it would increase chances to a select a performant peer in a safe manner

The test TestMisbehaviourProofsWrongBlockEpoch is non-deterministically failing in develop branch, which isn't related to these changes

uprendis commented 3 months ago

May I ask you to provide additional information on how this was tested so that it may be reproduced? Maybe also share collected data?

An easy way to test this is to copy a datadir and sync it up to a particular epoch via P2P using two different versions. Measure the time it takes for each version to sync. To stop at a specific epoch, we may use the --exitwhensynced.epoch flag

I will measure one or a few runs to gather some initial statistics. The results will vary between runs due to the different peer compositions each time. We would need to run it many-many times to make it look like a metric (which is doable and useful but beyond this PR)

uprendis commented 3 months ago

Epochs 5577-5800: Before: 113m37.286s After: 11m49.213s

Note that these are very old epochs - which means that very few nodes still have events from these epochs, which makes this heuristic more relevant than in latest epochs (and it changes after each new genesis), hence 10x improvement. That’s a topic for a separate discussion though - with some other long awaiting possible changes. Feel free to DM me for more details

thaarok commented 1 week ago

Replaced by #214