tendermint / tm-db

Common database interface for various database backends for Tendermint Core and Cosmos SDK

Apache License 2.0

89 stars 136 forks source link

Performance review #211

Closed faddat closed 2 years ago

faddat commented 2 years ago

goleveldb

https://youtu.be/Ho6DHkZax2c

badgerdb

https://youtu.be/tITLgGKDMDc

rocksdb

https://youtu.be/47PWHwkM0MY

All of notional's relay infrastructure has had to adopt rocksdb (osmosis, terra, gaia, juno) out of necessity. Given the overwhelming improvement, is there any reason not to work it into default workflows (slowly and in a non-jarring manner)

3xish is pretty good-- as is so far-- 100% compatibly.

Reasons not to rocksdb

not all go makes compiles not super fun:

https://github.com/cosmos/gorocksdb/pull/3

pebbledb? https://github.com/cockroachdb/pebble

faddat commented 2 years ago

https://github.com/cockroachdb/pebble/blob/e2a8aef27e1b170d78f466fb97fda036a693abab/batch.go#L418

https://github.com/cockroachdb/pebble/issues/1416

boz commented 2 years ago

3x on a drop-in-ish replacement is amazing @faddat!

I think that by defining a prefix extractor that partitions by store key prefixes, we may be able to get a big boost in query performance.

tac0turtle commented 2 years ago

Hey, it seems there is some increased performance in your video, but it would be good to get benchmarks and profiles to see the increased performance. Also I believe you would see similar performance across all dbs when they grow in size.

At the end, write performance is not a major issue in the sdk. We are working on changes in IAVL to improve read performance which will improve writes while state executing. This theory was put together through profiling.

faddat commented 2 years ago

@marbar3778 could we arrange a call or DM session to discuss relaying?

The issue is reads and I don't know how to properly quantify but this helps a great deal there, too.

I think that if you are following a hunch that IAVL changes will yeild bigger gains-- I think I agree. @ValarDragon and I have discussed this some, too.

tac0turtle commented 2 years ago

Happy to hop on a call!! this is helping push forward on things.

faddat commented 2 years ago

@boz -- @marbar3778 explained to me a bit more how we look for things in iavl, and I totally agree, those prefixes should help with performance a great deal.

faddat commented 2 years ago

This is superceded by #248