RCasatta / blocks_iterator

Iterate over bitcoin blocks
MIT License
51 stars 5 forks source link

remove rocksdb, use redb #67

Closed RCasatta closed 1 year ago

RCasatta commented 1 year ago

Unfortunately redb batch_write is much slower than rocksdb, I still like more redb because it doesn't take a life to compile and it's more typed. Probably adding redb without a feature and keeping feature gated rocksdb?

testnet stop @100_000 tmpfs                                                     

run | redb v1 | rocksdb                                                         
-----------------------                                                         
1   | 33s     | 9s                                                              
2   |  4s     | 5s                                                              

testnet stop @100_000 physical disk  NVMe                                           

              | durability       |                                              
run | redb v1 | None |  Eventual | rocksdb                                      
-------------------------------------------                                     
1   | 310s    |  58s |   308s    | 9s                                           
2   | 4s      |   4s |     5s    | 5s                                           

63

dpc commented 1 year ago

BTW. About the compile times. We used to have to wait forever for rocksdb to build in Fedimint, but then if you provide it a ROCKSDB_LIB_DIR to find a static c library, it can avoid building it. So that's what we do, and since our dev env is all nix flake based, it became non-issue.

RCasatta commented 1 year ago

thanks @dpc for letting me know that

RCasatta commented 1 year ago

close in favor of #69 (which doesn't replace rocksdb)