perf: flush import batches in parallel

[elias-orijtech]

As described in #664, both the construction and subsequent flushing of transaction batches are slow. This change is a minimal alternative to nothing else. This ensures simplicity and hopefully gains most of the performance improvement from #664.

On my macOS M1 machine:

goos: darwin goarch: arm64 pkg: github.com/cosmos/iavl BenchmarkImportBatch BenchmarkImportBatch-8 4 300864156 ns/op

running the same benchmark without this change:

BenchmarkImportBatch-8 3 353229292 ns/op

CC @odeke-em @yzang2019 @tac0turtle

[elias-orijtech]

@yzang2019 would something like this help your use-case in #664?

[tac0turtle]

any chance we can get a benchmark comparison of what is there now and this?

[elias-orijtech]

any chance we can get a benchmark comparison of what is there now and this?

Done.

[tac0turtle]

gentle ping on a benchmark for this work.

[elias-orijtech]

gentle ping on a benchmark for this work.

Added a benchmark as a larger version of the existing benchmark. See the commit https://github.com/cosmos/iavl/pull/793/commits/b9a01531e126d43d2b5920119961fe15c55af5e2 for details of before/after.

[tac0turtle]

oh sorry missed that thank you

[tac0turtle]

@mergifyio backport release/v1.x.x

[mergify[bot]]

backport release/v1.x.x

✅ Backports have been created

* [#820 perf: flush import batches in parallel (backport #793)](https://github.com/cosmos/iavl/pull/820) has been created for branch `release/v1.x.x`

[yzang2019]

Sei has done some benchmark and with a extremely large snapshot, we are able to reduce the import time by 2X with a large batch size and parallel flush