This PR implements #51, taking a large inspiration from the im-rs crate .
It uses a second heed3/Cargo.toml with the necessary dependencies. When you need to work or publish the heed3 crate, you need to cp heed3/Cargo.toml heed/Cargo.toml. The examples were moved out of the standard heed/examples/ folder to make them compile when working on both crates.
There are three crates now...
heed: the one you know. Based on LMDB 0.9 (the mdb.master branch).
heed3: Based on LMDB 1.0 (the mdb.master3 branch), without support for encryption, will eventually support checksumming.
heed3-encryption: The same as heed3 but with support for encryption.
...but I want to only have two
heed: the one you know. Based on LMDB 0.9 (the mdb.master branch).
heed3: Based on LMDB 1.0 (the mdb.master3 branch), with support for encryption through the EncryptedDatabase and EnvOpenOptions::open_encrypted type and method and eventually support checksumming.
Note that if we duplicate the code of the Database for the EncryptedDatabase type, we no longer need the following proc-macro as we will copy/paste and change the method signature, i.e., &RoTxn -> &mut RoTxn, by hand.
How it works?
By annotating all the heed methods that use a &RoTxn this way:
This way, we ensure that users do not keep pointers for potentially invalid bytes from LMDB between two get/put operations. It's a limitation of LMDB when you use the encryption feature. The LMDB pages are decrypted on the fly in a buffer that cycles. As a result, only a restricted amount of values' pointers are valid until the next operations.
To Do
[x] Modify the CI to test the heed3 crate.
[x] Create and use the lmdb-master3-sys based on the mdb.master3 branch.
[x] Introduce two EnvOpenOptions/EnvEntry?
[x] Retrieve the work from #134 for encryption support (and update the dependencies).
[ ] Add an explanation about the fact that there are three crates and why
[ ] In the README
[ ] In the documentation
[ ] Can we make the doc tests to pass, or be conditional on the branch?
[ ] Publish the lmdb-master3-sys and heed3 crates on crates.io.
[ ] Add the support for checksumming too (optional)
This PR implements #51, taking a large inspiration from the im-rs crate .
It uses a second heed3/Cargo.toml with the necessary dependencies. When you need to work or publish the heed3 crate, you need to
cp heed3/Cargo.toml heed/Cargo.toml
. The examples were moved out of the standard heed/examples/ folder to make them compile when working on both crates.There are three crates now...
heed
: the one you know. Based on LMDB 0.9 (themdb.master
branch).heed3
: Based on LMDB 1.0 (themdb.master3
branch), without support for encryption, will eventually support checksumming.heed3-encryption
: The same asheed3
but with support for encryption....but I want to only have two
heed
: the one you know. Based on LMDB 0.9 (themdb.master
branch).heed3
: Based on LMDB 1.0 (themdb.master3
branch), with support for encryption through theEncryptedDatabase
andEnvOpenOptions::open_encrypted
type and method and eventually support checksumming.Note that if we duplicate the code of the
Database
for theEncryptedDatabase
type, we no longer need the following proc-macro as we will copy/paste and change the method signature, i.e.,&RoTxn -> &mut RoTxn
, by hand.How it works?
By annotating all the heed methods that use a
&RoTxn
this way:It transforms the function signature to use a
&mut RoTxn
:This way, we ensure that users do not keep pointers for potentially invalid bytes from LMDB between two get/put operations. It's a limitation of LMDB when you use the encryption feature. The LMDB pages are decrypted on the fly in a buffer that cycles. As a result, only a restricted amount of values' pointers are valid until the next operations.
To Do
mdb.master3
branch.EnvOpenOptions/EnvEntry
?