Open edubart opened 1 month ago
This problem is also becoming more relevant when loading snapshots in the validator node on cheap cloud services that have just 1 CPU and 1GB ram, to the point loading a snapshot takes so long that the validator is unable to initialize properly due to timeouts and advances/inpects doesn't work.
Context
The emulator is starting to be used to create "reader nodes" in production for some dapps where it advances and saves a machine snapshot, then when an inspect comes the saved machine snapshot is loaded and an inspect request is served. The problem is that saving/loading could take minutes (for example when running in a cheap cloud service with just 2 CPU cores and 4GB of ram).
We could optimize all this loading/saving to be faster. The culprit here is root hash computation, most of time is spent there.
Possible solutions
One solution is to save the entire merkle tree state to a file in the snapshot, so not all merkle nodes have to be recomputed after loading. And also save checksum (like sha1sum) of all snapshot files to not lose integrity check, it should be faster to check a sha1sum of a few files rather than a merkle tree hash. Maybe even CRC64 could be enough, I think we don't need a very secure and heavy hash to just check snapshot file integrity.