search

jnwatson / py-lmdb

Universal Python binding for the LMDB 'Lightning' Database
http://lmdb.readthedocs.io/
Other
646 stars 106 forks source link

Update liblmdb, add building for 3.12 #361

Closed jnwatson closed 5 months ago

seydx commented 5 months ago

Hey @jnwatson,

thank you very much for py-lmdb and I'm glad to see that it's being updated again!

I'm wondering if these changes are compatible with lmdb-js (which uses the current code of LMDB version, 0.9.90)

Regarding the version of LMDB, I couldn't figure it out. However, the last release seems to be 0.9.33? Maybe you can help me out here and tell me if 0.9.31 is compatible with LMDB 0.9.90?

jnwatson commented 4 months ago

Ugh. It looks like there are multiple forks of LMDB floating around.

I try to keep sync'd with released tags of the github mirror: https://github.com/LMDB/lmdb/tree/LMDB_0.9.31.

Looking at the repo it is supposed to mirror from, it turns out there's a later tagged release of 0.9.33 from a few weeks ago: https://git.openldap.org/openldap/openldap/-/tags/LMDB_0.9.33. Why that hasn't propagated to the mirror is unknown.

Now, there's a master3 branch on the OpenLDAP repo that has 0.9.70, but it isn't released. That's the version in Node-LMDB. https://www.npmjs.com/package/node-lmdb?activeTab=code

It seems like lmdb-js has its own fork of LMDB. It has changes that I don't see the origin of. I really can't speak to compatibility there.

I do have some minor changes to LMDB that I keep in a patch file, but those are at the periphery and definitely don't impact the on-disk format. (They are for live-backup).

So far, in the released tags of upstream LMDB, there has been no data format change.

In other words, py-lmdb is compatible with released versions of upstream LMDB. You'll have to ask the maintainer of lmdb-js whether it is as well.

seydx commented 4 months ago

There is an old github issue on lmdb-js, where the maintainer of lmdb-js describes which version he use and why.

https://github.com/kriszyp/lmdb-js/issues/193

Maybe @kriszyp can leave a short comment here? Using lmdb-js together with py-lmdb would help a lot people and projects

jnwatson commented 4 months ago

Thanks for the info. While not super duper formal, OpenLDAP/Symas does have tagged releases, and large distributions use those for packaging, e.g. https://launchpad.net/ubuntu/oracular/amd64/liblmdb-dev/0.9.31-1build1. In fact you can use those libraries and build py-lmdb directly against those libraries and they will work.

The v2 data format isn't technically frozen. The mdb.master3 is where future work is going, but it isn't versioned. It will be a while before it makes sense to add support for this. You might try building py-lmdb manually and seeing if it builds.

kriszyp commented 4 months ago

I will note that lmdb-js does preserve the ability to create a build for the release/v1 version of LMDB to facilitate compatibility with other libraries (like this one).