mozilla / rkv

A simple, humane, typed key-value storage solution.
https://crates.io/crates/rkv
Apache License 2.0
310 stars 53 forks source link

provide mechanism for consumers to migrate DB files between 32- and 64-bit builds #136

Closed mykmelez closed 5 years ago

mykmelez commented 5 years ago

rkv should provide some mechanism for consumers to migrate database files between 32-bit and 64-bit builds, since LMDB 0.9 files are bit-width-dependent, and users sometimes switch between 32-bit and 64-bit builds of software that uses rkv or copy database files from a 32-bit to a 64-bit system.

This blog post about The LMDB file format notes that it's possible to compile a 32-bit build of mdb_dump and use that on a 64-bit system to dump a 32-bit database file to a portable format that can be reloaded into a new file. It also references this Lua reimplementation of the mdb_dump utility that can read 32-bit database files on a 64-bit system (and presumably vice-versa).

Note that the database format on the LMDB master branch (i.e. the development version that is slated to become LMDB 1.0) is bit-width-independent, so this issue won't exist there. But if we upgrade rkv to LMDB master/1.0 in the future, then we'll have to deal with database migration from the 0.9 format to the 1.0 format. So we can't avoid this issue by upgrading the version of LMDB we embed.

ncloudioj commented 5 years ago

Closed via abd08f409aac4d8ff3d537e5ad8887590c44171d