Support concurrency for IAVL and fix Racing conditions

mattverse commented 1 year ago

cref: https://github.com/cosmos/iavl/issues/696

This PR fixes the problem stated above by replacing maps that causes racing conditions to sync map to be thread safe.

Confirmed and tested on a node with the following procedure:

Have node running
Spam it with tx bots and query bots
Previously node would crash within 1, maximum 2 minutes with the racing condition logs mentioned above
With fix applied node would not crash for 40+ minutes.

After 40+ minutes, node does seem to crash, but suspecting this may be to different concurrency issues (still doing more investigation)

julienrbrt commented 1 year ago

I think this should target master instead of the release branch, this issue is probably present on v0.20 as well and v1.0.0.

mattverse commented 1 year ago

@julienrbrt We can also backport this to older release versions as well though right?

mattverse commented 1 year ago

@julienrbrt Awesome, let's do this: I'll close this issue for now and create a PR pointing at main. After we have that reviewed we can probably backport this to all necessary branches

julienrbrt commented 1 year ago

Awesome, let's do that!

cosmos / iavl

Support concurrency for IAVL and fix Racing conditions #802