search

cometbft / cometbft-db

Database wrapper for CometBFT
Apache License 2.0
29 stars 51 forks source link

goleveldb: potential improvements #102

Closed melekes closed 1 month ago

melekes commented 11 months ago

If goleveldb is picked as the primary DB for CometBFT, we should identify any potential improvements we can make.

1. Disable seek compaction

Multiple reports turning off seek compaction improves the compaction process:

https://github.com/google/leveldb/issues/857 https://github.com/klaytn/klaytn/pull/715 https://github.com/ethereum/go-ethereum/pull/20130

2. Pre-allocate memory for a batch

goleveldb could be more efficient when growing the batch buffer at once. It is better to pre-allocate the buffer.

3. Investigate changes made by Finschia team

https://github.com/Finschia/tm-db/commits/main

faddat commented 10 months ago

tbh everyone is moving to pebble and goleveldb is no longer supported by its author and adds a replace statement to everything.

Since performance issues stemming from comet are affecting numerous mainnets, can we just go with what we know is supported and performant?

melekes commented 10 months ago

tbh everyone is moving to pebble and goleveldb is no longer supported by its author and adds a replace statement to everything.

Thanks for the insight 👍 Who's moving to pebble? Please comment on https://github.com/cometbft/cometbft/issues/1039, which is more relevant to this discussion.

faddat commented 10 months ago

I thought that geth recently announced a move to pebble?

What you're saying here makes it seem maybe not though

melekes commented 9 months ago

worth noting that https://github.com/syndtr/goleveldb is unmaintained