search

filecoin-project / lotus

Reference implementation of the Filecoin protocol, written in Go
https://lotus.filecoin.io/
Other
2.83k stars 1.26k forks source link

Fix performance of `StateCompute` RPC on Archival nodes #12294

Open aarshkshah1992 opened 2 months ago

aarshkshah1992 commented 2 months ago

StateCompute for an epoch near the head can take 1 minute to complete on an archival node. For epochs far in the past, it can 4-5 minutes. This is only for a single request and performance degrades further for multiple requests. This makes it extremely hard to build a Filecoin explorer(which needs archived state) as it can take months to index the entire chain.

We should get access to an archival node and profile the StateCompute RPC API to find performance hotspots and fix them.

(This was learned when talking with Zondax on 2024-07-24.)

dubuqingfeng commented 1 month ago

Is there a solution for this?

rvagg commented 1 month ago

Not yet, we're not entirely sure what the issue might be other than badger being really bad at large amounts of data and probably really bad at data that's not touched very much (historical).