Closed hannahhoward closed 3 years ago
Thanks for the issue and insight.
I think we can probably reach some valuable low-hanging fruit quickly with approach (ii). That will be much faster to execute in addition to offering more control. As you point out, exerting that control will require effort as we try to make a chain that's more representative of a real network. As an opinion, I suggest wrapping the existing chain.Builder with "application-level" functionality, rather than coupling it directly with specific actor methods etc.
Our work so far points to state tree storage and serialisation, and we can give that a good workout without invoking the complexities of real proof calculation or validation required for a more representative network (and when data do point to those latter as bottlenecks, they can be benchmarked independently.
Something like approach (i) will still be valuable, but I think a bit later.
Description
As we work on improving ChainSync performance, we need a benchmark we can run to track our improvements.
What we have so far is:
We need benchmarks that sits somewhere between "testing a very small part of the process" and "chainsync is slow after devnet has been running three months". Such benchmarks could cover an entire chainsync process including network fetch time, or they could test some small but sizable part of the process.
Acceptance criteria
An actual Golang benchmark we can run, repeated, maybe eventually as part of CI (but far off on that one) that will give us an approximate idea of how long it will take to sync a sizable, real world chain. Ideally we could test against multiple types of chains with various properties
Risks + pitfalls
The biggest obstacle to benchmarks is we have no way to quickly produce a real block chain that simulates what real transactions operating on a real FileCoin network would look like. (or even produce such any such chain in repeatable manner)
There's the additional risk of spending so much time on this problem that we don't get to actually improving things :)
Where to begin
It seems like there are two potential routes for tackling the big problem, which is making chains: