Closed elias-orijtech closed 1 year ago
@yzang2019 would something like this help your use-case in #664?
any chance we can get a benchmark comparison of what is there now and this?
any chance we can get a benchmark comparison of what is there now and this?
Done.
gentle ping on a benchmark for this work.
gentle ping on a benchmark for this work.
Added a benchmark as a larger version of the existing benchmark. See the commit https://github.com/cosmos/iavl/pull/793/commits/b9a01531e126d43d2b5920119961fe15c55af5e2 for details of before/after.
oh sorry missed that thank you
@mergifyio backport release/v1.x.x
backport release/v1.x.x
Sei has done some benchmark and with a extremely large snapshot, we are able to reduce the import time by 2X with a large batch size and parallel flush
As described in #664, both the construction and subsequent flushing of transaction batches are slow. This change is a minimal alternative to nothing else. This ensures simplicity and hopefully gains most of the performance improvement from #664.
On my macOS M1 machine:
goos: darwin goarch: arm64 pkg: github.com/cosmos/iavl BenchmarkImportBatch BenchmarkImportBatch-8 4 300864156 ns/op
running the same benchmark without this change:
BenchmarkImportBatch-8 3 353229292 ns/op
CC @odeke-em @yzang2019 @tac0turtle