optakt / flow-dps

Flow Data Provisioning Service
Apache License 2.0
29 stars 13 forks source link

Live: improve downloader performance #411

Closed awfm9 closed 3 years ago

awfm9 commented 3 years ago

Goal of this PR

This PR increases the size of the downloader's buffer, so it will keep polling more blocks in the background than it otherwise would have. At the same time, this means that a single Next() call with an empty buffer would wait a long time to return (until the entire poll is finished). It was already slow with 8 instead of 100. So with this change, it will never wait for the buffer to be full, but instead return to the business logic which will have a much smaller wait. Combined, these changes should significantly reduce initial sync time.

Checklist