search

RCasatta / blocks_iterator

Iterate over bitcoin blocks
MIT License
51 stars 5 forks source link

add instruction for dynamic rocksdb #78

Closed RCasatta closed 6 months ago

RCasatta commented 1 year ago 
          BTW. About the compile times. We used to have to wait forever for rocksdb to build in Fedimint, but then if you provide it a `ROCKSDB_LIB_DIR` to find a static c library, [it can avoid building it](https://github.com/rust-rocksdb/rust-rocksdb/blob/6e19f1da84633ba42b69fdfc7e74b72d19f92901/librocksdb-sys/build.rs#L350). So that's what we do, and since our dev env is all nix flake based, it became non-issue.

_Originally posted by @dpc in https://github.com/RCasatta/blocks_iterator/issues/67#issuecomment-1474042625_

RCasatta commented 1 year ago

In #79 I tried rocksdb dynamic linking with distro-provided rocksdb (sudo apt install librocksdb-dev libsnappy-dev) with great compile-time results but very disappointing bench results:

ROCKSDB_LIB_DIR=/usr/lib/ cargo +nightly bench 
...
    Finished bench [optimized] target(s) in 36.93s
...
test bench_db_batch                    ... bench:  42,473,786 ns/iter (+/- 4,740,588)
test bench_db_no_batch                 ... bench:  42,864,041 ns/iter (+/- 3,682,106)
cargo +nightly bench 
...
    Finished bench [optimized] target(s) in 4m 12s
...
test bench_db_batch                    ... bench:   1,090,659 ns/iter (+/- 35,234)
test bench_db_no_batch                 ... bench:   1,901,121 ns/iter (+/- 116,950)

of interests: @dpc

dpc commented 1 year ago

40x ? WTH :D

dpc commented 1 year ago

It's hard to explain such a huge difference by even bad FFI alone. Are the versions being used the same, etc.? I mean 40ms per iteration seems more like some IO issue level of difference.

RCasatta commented 1 year ago

Are the versions being used the same, etc.?

No, however, I don't think such a difference could be associated only to version differences...

RCasatta commented 6 months ago

Nix users now do dynamic linking via the flake.nix configurations. It's activated for both development and also for the released version because benches now performs the same so it was probably some issue in that version