Create two different libraries: heed and heedx

[Kerollmops]

As a Reddit user pointed using a feature flag to make heed work with weither LMDB or MDBX is not the best way to do so. I did this because it was the easiest way to keep the same code base and only tune some functions.

Recently I found a potential alternative to publish two libraries with two different but keep the same code base.

#[cfg(pkg_name = "bonjour-hello")]
fn main() {
    println!("bonjour-hello");
}

#[cfg(pkg_name = "bonjour")]
fn main() {
    println!("bonjour");
}

#[cfg(pkg_name = "hello")]
fn main() {
    println!("hello");
}

bonjour-hello; RUSTFLAGS='--cfg pkg_name="bonjour"' cargo run
   Compiling bonjour-hello v0.1.0 (/Users/clementrenault/Documents/bonjour-hello)
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `target/debug/bonjour-hello`
bonjour

bonjour-hello; RUSTFLAGS='--cfg pkg_name="hello"' cargo run
   Compiling bonjour-hello v0.1.0 (/Users/clementrenault/Documents/bonjour-hello)
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `target/debug/bonjour-hello`
hello

bonjour-hello; RUSTFLAGS='--cfg pkg_name="bonjour-hello"' cargo run
   Compiling bonjour-hello v0.1.0 (/Users/clementrenault/Documents/bonjour-hello)
    Finished dev [unoptimized + debuginfo] target(s) in 0.25s
     Running `target/debug/bonjour-hello`
bonjour-hello

I am here to mentor anyone who is interrested in helping me fixing this issue, and if someone would like to take a look of where the action takes place, he/she can look at the mdb/mod.rs file, this is, in part, where changes will be make.

[repi]

Would indeed be cleaner with separate crates for them I think, features don't work that well for mutually exclusive stuff.

One could have a deep transitive dependency that brings in heed with the mdbx feature that then silently switches the compilation for ones entire project to use that. (probably not too common in practice, but not very explicit)

[Kerollmops]

There is news about this way of doing things, I will take a look at the im and im-rc crates from bodil.

[jonsecchis]

When I come to think about it, from a maintenance point of view, providing two back-ends in one repository seems like a liability (burden) to me, regardless of the mechanics. It would be a great loss for the libmdbx project to not have Heed as a Rust wrapper, but maybe someone can make a fork and maintain the wrapper in an entirely independent manner. That would allow for both projects to be more robust, lean, focused and free to grow on its own.

MeiliSearch being the main driver for this project and it not using libmdbx as a back-end also corroborates to my argument that it is indeed a liability to maintain one back-end that lacks the necessity and the resources.

Consider how the issue tracker for Heed would look in a future wherelibmdbx takes a different path in a next major version? It is sort of conflicting interests. libmdbx being modeled after liblmdb does not mean that they both should forever be API-compatible and share the same goals. Much like liblmdb was modeled after berkleydb and both have nothing to do with each other. So maybe their similarities are just temporal, or accidental.

[Kerollmops]

Reviving this issue because I would like to create two crates. One of the crate is using mdb.master and the other mdb.master3.

The one using mdb.master3 will support encryption with the EnvOpenOptions::encrypt_with and therefore it's not allowed to keep the pointers returned by the functions in between two read or write calls. So the functions signature must be something around the following lines even if no modifications can happen. This is the way, in Rust, to disallow using a slice with a lifetime after this same lifetime is modified you can declare it mut.

fn get<'txn>(&Database, rtxn: &'txn mut RoTxn, key: &'_ [u8]) -> &'txn [u8];