search

romanz / electrs

An efficient re-implementation of Electrum Server in Rust
MIT License
1.02k stars 374 forks source link

Would it be possibe to swap rocksdb for something else? #201

Open Kixunil opened 4 years ago

Kixunil commented 4 years ago

I found that it takes enormous time to build and strangely, it's now taking me several hours.

It'd be nice if it could be swapped for something else somehow. Even if its slightly slower.

romanz commented 4 years ago

RocksDB build time is indeed an issue - maybe we can try LMDB? It builds significantly faster :)

$ time cargo build --release
   Compiling cc v1.0.47
   Compiling pkg-config v0.3.17
   Compiling libc v0.2.66
   Compiling bitflags v1.2.1
   Compiling lmdb-sys v0.8.0 (/media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys)
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c: In function ‘mdb_env_get_maxkeysize’:
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:10033:33: warning: unused parameter ‘env’ [-Wunused-parameter]
warning:  mdb_env_get_maxkeysize(MDB_env *env)
warning:                         ~~~~~~~~~^~~
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c: In function ‘mdb_cursor_put’:
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:6737:9: warning: this statement may fall through [-Wimplicit-fallthrough=]
warning:       if (SIZELEFT(fp) < offset) {
warning:          ^
warning: /media/roman/data/roman/Code/Rust/lmdb-rs/lmdb-sys/lmdb/libraries/liblmdb/mdb.c:6742:5: note: here
warning:      case MDB_CURRENT:
warning:      ^~~~
   Compiling lmdb v0.8.0 (/media/roman/data/roman/Code/Rust/lmdb-rs)
    Finished release [optimized] target(s) in 4.94s

real    0m4.952s
user    0m12.722s
sys 0m0.429s
romanz commented 4 years ago

OTOH, there is https://github.com/spacejam/sled which is pure Rust :)

Kixunil commented 4 years ago

LMDB sounds good. Sled is beta, so I think we should wait with that.

Hopefully I will find some time to contribute a PR until the end of January.

GeneFerneau commented 2 years ago

LMDB sounds good. Sled is beta, so I think we should wait with that.

2021 and sled is still beta

Just my two cents, maybe a pluggable datastore API makes sense (if not too much extra work). What are your thoughts? Is anyone currently working on this?

Kixunil commented 2 years ago

Compile time is partially solved by dynamic linking (and sufficient for me so far) but having change-able database could still be interesting. It should be possible using Clean Architecture, not sure how much work.

GeneFerneau commented 2 years ago

Not familiar with Clean Architecture (looks it up).

SQLite might be another good alternative

romanz commented 2 years ago

https://crates.io/crates/heed (Rust LMDB/MDBX wrapper) is used by https://github.com/meilisearch/MeiliSearch/pull/230:

$ cargo build --release
    Updating crates.io index
  Downloaded pkg-config v0.3.20
  Downloaded lmdb-rkv-sys v0.11.0
  Downloaded bytemuck_derive v1.0.1
  Downloaded synchronoise v1.0.0
  Downloaded crossbeam-queue v0.1.2
  Downloaded 5 crates (219.0 KB) in 1.07s
   Compiling proc-macro2 v1.0.30
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.80
   Compiling serde_derive v1.0.130
   Compiling serde v1.0.130
   Compiling libc v0.2.104
   Compiling ryu v1.0.5
   Compiling cfg-if v0.1.10
   Compiling lazy_static v1.4.0
   Compiling pkg-config v0.3.20
   Compiling serde_json v1.0.68
   Compiling cc v1.0.71
   Compiling itoa v0.4.8
   Compiling heed-traits v0.7.0 (/media/roman/3TB/roman/Code/Rust/heed/heed-traits)
   Compiling byteorder v1.4.3
   Compiling once_cell v1.8.0
   Compiling crossbeam-utils v0.6.6
   Compiling crossbeam-queue v0.1.2
   Compiling synchronoise v1.0.0
   Compiling lmdb-rkv-sys v0.11.0
   Compiling quote v1.0.10
   Compiling page_size v0.4.2
   Compiling bytemuck_derive v1.0.1
   Compiling bytemuck v1.7.2
   Compiling bincode v1.3.3
   Compiling heed-types v0.7.2 (/media/roman/3TB/roman/Code/Rust/heed/heed-types)
   Compiling heed v0.10.6 (/media/roman/3TB/roman/Code/Rust/heed/heed)
    Finished release [optimized] target(s) in 13.88s
romanz commented 2 years ago

http://www.lmdb.tech/bench/microbench/ looks promising :)

prusnak commented 2 years ago

FWIW there are several crates implementing KV storage via LMDB, with different levels of abstraction:

bukatea commented 2 years ago

Is there any recent update on this? I'd like to use this crate alongside another crate that depends on an incompatible version of rocksdb

Kixunil commented 2 years ago

@bukatea it's not simple, so if you're in a hurry you're probably better off resolving the incompatibility.

bukatea commented 2 years ago

@Kixunil thx much, happened to find another solution :)

romanz commented 1 year ago

https://github.com/cberner/redb can be an option as well.

romanz commented 11 months ago

https://twitter.com/sadisticsystems/status/1684906383227961344 -> https://docs.rs/sled/1.0.0-alpha.1/sled

mikedilger commented 6 days ago

I've been using LDMB via https://docs.rs/heed/latest/heed/ for about a year in a nostr client and I'm very happy with it. Heed was recommended to me by Howard Chu the LMDB author as the best rust interface to it.

But you should know the downsides. The database doesn't automatically compress itself, and copy on write with multiple transactions can leave data larger than it needs to be. Using the 'mdb_copy -c' command line resolves this, but there is no rust library call to do this compaction (yet, IIRC).

Sled always looked like a cool idea for optimizing high-contention situations, but too experimental.

I haven't used rocksdb yet so I can't say if LMDB seems better or worse to me.

Kixunil commented 6 days ago

FYI since opening this issue I'm less enthusiastic about it. A pure Rust impl would be statically linked which has its own downsides. It's cool how much compile time gets reduced when dynamically linking. (I exclusively link dynamically.)

antonilol commented 5 days ago

Even when linking dynamically, the library still has to be compiled (not talking about downloading precompiled libraries here, you can statically link them too then). When I compile electrs (statically) after a code change, rocksdb and all the other crates wont have to be recompiled, simply because of caching. I do agree that dynamic linking is good for disk usage on a system by not duplicating libraries in every binary (I use Arch Linux myself!).

LMDB can be dynamically linked too but LMDB compiles much faster (few seconds!) than RocksDB anyway given the amount of code they each have.

Kixunil commented 4 days ago

@antonilol I don't compile rocksdb myself, I use the system one. And caching doesn't help when doing Debian packages with deterministic compilation since it's best to build them from scratch to avoid any surprises.