make pebble the clear choice

faddat commented 10 months ago

Closes: #90

add pebbledb
use latest pebble
added a ton of test rigor and fixed linting
if always false we don't need the param
let's go fast

faddat commented 10 months ago

All I can say is:

informal never merged it
ICF never funded it
notional never optimized it: most interactions with informal are disincentivized
we believe and can sense / smell that Anton could be different.
we are gonna try again.
Jacob thinks Osmosis will benefit from optimizing it, so -- it is being optimized.

codecov[bot] commented 10 months ago

Codecov Report

Attention: 97 lines in your changes are missing coverage. Please review.

Comparison is base (b15320b) 77.53% compared to head (070a044) 73.98%.

Additional details and impacted files

[![Impacted file tree graph](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115/graphs/tree.svg?width=650&height=150&src=pr&token=QcxCp5s2sq&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft)](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft) ```diff @@ Coverage Diff @@ ## main #115 +/- ## ========================================== - Coverage 77.53% 73.98% -3.55% ========================================== Files 22 13 -9 Lines 1785 1134 -651 ========================================== - Hits 1384 839 -545 + Misses 342 250 -92 + Partials 59 45 -14 ``` | [Files](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft) | Coverage Δ | | |---|---|---| | [goleveldb.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-Z29sZXZlbGRiLmdv) | `59.84% <100.00%> (ø)` | | | [test\_helpers.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-dGVzdF9oZWxwZXJzLmdv) | `100.00% <100.00%> (ø)` | | | [util.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-dXRpbC5nbw==) | `36.66% <100.00%> (ø)` | | | [db.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-ZGIuZ28=) | `35.00% <50.00%> (ø)` | | | [memdb\_iterator.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-bWVtZGJfaXRlcmF0b3IuZ28=) | `90.42% <33.33%> (-2.99%)` | :arrow_down: | | [memdb.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-bWVtZGIuZ28=) | `69.64% <16.66%> (-1.27%)` | :arrow_down: | | [pebble.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-cGViYmxlLmdv) | `71.93% <71.93%> (ø)` | | ... and [10 files with indirect coverage changes](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft) | [Files](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft) | Coverage Δ | | |---|---|---| | [goleveldb.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-Z29sZXZlbGRiLmdv) | `59.84% <100.00%> (ø)` | | | [test\_helpers.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-dGVzdF9oZWxwZXJzLmdv) | `100.00% <100.00%> (ø)` | | | [util.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-dXRpbC5nbw==) | `36.66% <100.00%> (ø)` | | | [db.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-ZGIuZ28=) | `35.00% <50.00%> (ø)` | | | [memdb\_iterator.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-bWVtZGJfaXRlcmF0b3IuZ28=) | `90.42% <33.33%> (-2.99%)` | :arrow_down: | | [memdb.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-bWVtZGIuZ28=) | `69.64% <16.66%> (-1.27%)` | :arrow_down: | | [pebble.go](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft#diff-cGViYmxlLmdv) | `71.93% <71.93%> (ø)` | | ... and [10 files with indirect coverage changes](https://app.codecov.io/gh/cometbft/cometbft-db/pull/115/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cometbft)

faddat commented 10 months ago

faddat@Whites-MacBook-Pro cometbft-db % GOARCH=arm64 go test -bench=BenchmarkPebbleDBvsGoLevelDB goos: darwin goarch: arm64 pkg: github.com/cometbft/cometbft-db BenchmarkPebbleDBvsGoLevelDB/PebbleDB-8 322890 3930 ns/op BenchmarkPebbleDBvsGoLevelDB/GoLevelDB-8 315639 4425 ns/op PASS ok github.com/cometbft/cometbft-db 7.470s

faddat commented 10 months ago

faddat@Whites-MacBook-Pro cometbft-db % GOARCH=arm64 go test -bench=BenchmarkPebbleDBvsGoLevelDB goos: darwin goarch: arm64 pkg: github.com/cometbft/cometbft-db BenchmarkPebbleDBvsGoLevelDB/PebbleDB-8 348592 3866 ns/op BenchmarkPebbleDBvsGoLevelDB/GoLevelDB-8 307240 4261 ns/op PASS ok github.com/cometbft/cometbft-db 7.409s

faddat commented 10 months ago

However if tested against iavl, this would be even more superieor.

faddat commented 10 months ago

Please use this mainly for the benchmarks as I think I made a kind of significant finding here with regard to high bandwidth situations.

faddat commented 10 months ago

TO: Maintainer -

please run the benchmarks I think they could explain some recent stuff.

faddat commented 10 months ago

I'll pull benchmarks from here.

faddat commented 9 months ago

to: maintiner (idk who this is) please run benchmarks on a solidly fast machine.

I fear if we don't it wont match the real world.

faddat commented 9 months ago

In general the benchmarks will exceed what a normal computer can do. my 64gb macbook pro handles it well. I could reduce the size of values, which I am somewhat guessing on, and increase the number of keys. The critical thing to note is that

go test -bench=.
goos: darwin
goarch: arm64
pkg: github.com/cometbft/cometbft-db
BenchmarkGoLevelDBRandomReadsWrites-12            374047              3878 ns/op
BenchmarkMemDBRangeScans1M-12                       1087           1122654 ns/op
BenchmarkMemDBRangeScans10M-12                      1060           1126530 ns/op
BenchmarkMemDBRandomReadsWrites-12               1000000              2080 ns/op
BenchmarkDB/Keys_10000_Values_512_pebbledb-12           1000000000               0.03704 ns/op
BenchmarkDB/Keys_10000_Values_512_goleveldb-12          1000000000               0.05867 ns/op
BenchmarkDB/Keys_10000_Values_1024_pebbledb-12          1000000000               0.03695 ns/op
BenchmarkDB/Keys_10000_Values_1024_goleveldb-12         1000000000               0.08461 ns/op
BenchmarkDB/Keys_10000_Values_2048_pebbledb-12          1000000000               0.03693 ns/op
BenchmarkDB/Keys_10000_Values_2048_goleveldb-12         1000000000               0.1145 ns/op
BenchmarkDB/Keys_10000_Values_4096_pebbledb-12          1000000000               0.04425 ns/op
BenchmarkDB/Keys_10000_Values_4096_goleveldb-12         1000000000               0.2443 ns/op
BenchmarkDB/Keys_25000_Values_512_pebbledb-12           1000000000               0.08316 ns/op
BenchmarkDB/Keys_25000_Values_512_goleveldb-12          1000000000               0.1817 ns/op
BenchmarkDB/Keys_25000_Values_1024_pebbledb-12          1000000000               0.07789 ns/op
BenchmarkDB/Keys_25000_Values_1024_goleveldb-12         1000000000               0.2394 ns/op
BenchmarkDB/Keys_25000_Values_2048_pebbledb-12          1000000000               0.07354 ns/op
BenchmarkDB/Keys_25000_Values_2048_goleveldb-12         1000000000               0.3182 ns/op
BenchmarkDB/Keys_25000_Values_4096_pebbledb-12          1000000000               0.07995 ns/op
BenchmarkDB/Keys_25000_Values_4096_goleveldb-12                1        1674754750 ns/op
BenchmarkDB/Keys_50000_Values_512_pebbledb-12           1000000000               0.1466 ns/op
BenchmarkDB/Keys_50000_Values_512_goleveldb-12          1000000000               0.3238 ns/op
BenchmarkDB/Keys_50000_Values_1024_pebbledb-12          1000000000               0.1558 ns/op
BenchmarkDB/Keys_50000_Values_1024_goleveldb-12         1000000000               0.4517 ns/op
BenchmarkDB/Keys_50000_Values_2048_pebbledb-12          1000000000               0.1485 ns/op
BenchmarkDB/Keys_50000_Values_2048_goleveldb-12         1000000000               0.7940 ns/op
BenchmarkDB/Keys_50000_Values_4096_pebbledb-12          1000000000               0.1651 ns/op
BenchmarkDB/Keys_50000_Values_4096_goleveldb-12                1        1899533750 ns/op
BenchmarkDB/Keys_75000_Values_512_pebbledb-12           1000000000               0.2108 ns/op
BenchmarkDB/Keys_75000_Values_512_goleveldb-12          1000000000               0.4588 ns/op
BenchmarkDB/Keys_75000_Values_1024_pebbledb-12          1000000000               0.2225 ns/op
BenchmarkDB/Keys_75000_Values_1024_goleveldb-12         1000000000               0.7332 ns/op
BenchmarkDB/Keys_75000_Values_2048_pebbledb-12          1000000000               0.2192 ns/op
BenchmarkDB/Keys_75000_Values_2048_goleveldb-12                1        1429007458 ns/op
BenchmarkDB/Keys_75000_Values_4096_pebbledb-12          1000000000               0.2486 ns/op
BenchmarkDB/Keys_75000_Values_4096_goleveldb-12                1        3416400792 ns/op
BenchmarkDB/Keys_100000_Values_512_pebbledb-12          1000000000               0.2646 ns/op
BenchmarkDB/Keys_100000_Values_512_goleveldb-12         1000000000               0.6500 ns/op
BenchmarkDB/Keys_100000_Values_1024_pebbledb-12         1000000000               0.3111 ns/op
BenchmarkDB/Keys_100000_Values_1024_goleveldb-12               1        1034177375 ns/op
BenchmarkDB/Keys_100000_Values_2048_pebbledb-12

pebble is in general many times faster.

Ulike goleveldb, pebble is maintained.

faddat commented 9 months ago

sometimes pebble is a few million times faster.

Flakiness, anyone?

faddat commented 9 months ago

the big worry here is that goleveldb does not act in a predictable manner.

faddat commented 9 months ago

preparing a branch of cometbft's main to see if we can eliminate time based flakiness with pebble.

adizere commented 9 months ago

I'm a fan. How is pebbleDB handling workload in Berachain?

Really wish you didn't do unnecessary changes here so we could review it easier. But aside from that, concept ACK and we'll review this ASAP, thanks for patience!

melekes commented 9 months ago

Closing in favor of https://github.com/cometbft/cometbft-db/pull/112. Let's focus on a single PR. After it's merged, we can do linting.

cometbft / cometbft-db

make pebble the clear choice #115

Codecov Report