read-txn-no-tls not working as expected

[emuellen]

Hello, I want to use the same read-transaction across several threads in my application. I activated the feature in my Cargo.toml as follows:

heed = { version = "0.20", default-features = false, features = [
    "read-txn-no-tls",
    "serde",
    "serde-bincode",
    "serde-json",
] }

This is the code snippet in question:

        let txn = lmdb_handle.environment.read_txn().map_err(|e| Status::internal(format!("Couldn't launch cache read transaction: {}", e)))?;

        let result: Vec<_> = prefixes
            .par_chunks(chunk_size)
            .map(|chunked_prefixes| {
                for prefix in chunked_prefixes {
                    let gallery = db
                        .prefix_iter(&txn, &format!("{:02x}", prefix))
                        .map_err(|e| Status::internal(format!("Failed to fetch prefix {prefix:02x} from cache: {}", e)))?;
                   ...
              }
        }).collect();

Now, the compiler returns this error:

`*mut lmdb_master_sys::MDB_txn` cannot be shared between threads safely
within `{closure@...:96:18: 96:36}`, the trait `Sync` is not implemented for `*mut lmdb_master_sys::MDB_txn`, which is required by `{closure@...:96:18: 96:36}: Sync`
required because it appears within the type `&RoTxn<'_>`

Am I doing something wrong or is it because Send is not implemented for lmdb_master_sys::MDB_txn ?

Thanks a lot for your help!

[Kerollmops]

Hey @emuellen 👋

I want to use the same read-transaction across several threads in my application.

This is not possible and is disallowed by LMDB. You can find more information about this cargo feature in the TOML. To be able to read your database from different threads I recommend you to open one transaction by thread, read transactions can be moved between thread when the read-txn-no-tls feature is enabled (RoTxn: Send) but will never be Sync.

Please look at my blog post about LMDB and Meilisearch and the different safe tricks and techniques you can use.

Have a nice day 🧇