kriszyp / lmdb-js

Simple, efficient, ultra-fast, scalable data store wrapper for LMDB
Other
486 stars 39 forks source link

`keyIsUint32` option of a database seems to be applied to the whole store #42

Closed vladar closed 3 years ago

vladar commented 3 years ago

I get the following error when trying to use keyIsUint32 option:

Error: In database data: Invalid key. Should be an unsigned 32-bit integer. (Specified with env.openDbi)

The example to reproduce it:

const { open } = require("lmdb-store")

const rootDb = open({
  name: `root`,
  path: `./data`,
  keyIsUint32: false,
})

const intKeys = rootDb.openDB({
  name: `intKeys`,
  keyIsUint32: true,
})

const strKeys = rootDb.openDB({
  name: `strKeys`,
  keyIsUint32: false,
})

async function run() {
  let lastPromise
  for (let intKey = 0; intKey < 100; intKey++) {
    const strKey = `k${intKey}`
    intKeys.put(intKey, `${intKey}-value`)
    lastPromise = strKeys.put(strKey, `${strKey}-value`)
  }
  await lastPromise
}

run().catch(console.error)
vladar commented 3 years ago

Interestingly, if I first write to strKeys and then to intKeys - the error is gone:

    strKeys.put(strKey, `${strKey}-value`)
    lastPromise = intKeys.put(intKey, `${intKey}-value`)
vladar commented 3 years ago

Then if I try to read data from this store (after the workaround) I get unexpected keys. I expect sequential numbers from 0 to 99 but getting this:

16 0-value
20 2-value
276 4-value
532 8-value
788 16-value
1044 32-value
1300 64-value
65299 1-value
263444 65-value
525332 33-value
525588 66-value
787732 67-value
1049364 17-value
1049620 34-value
1049876 68-value
1312020 69-value
1573908 35-value
1574164 70-value
1836308 71-value
2097684 9-value

The reading script:

const { open } = require("lmdb-store")

const rootDb = open({
  name: `root`,
  path: `./data`,
  keyIsUint32: false,
})

const intKeys = rootDb.openDB({
  name: `intKeys`,
  keyIsUint32: true,
})

const strKeys = rootDb.openDB({
  name: `strKeys`,
  keyIsUint32: false,
})

async function run() {
  intKeys.getRange({ snapshot: false }).forEach(({ key, value }) => {
    console.log(key, value)
  })
}

run().catch(console.error)

Not sure if I am missing something or this is a bug.

kriszyp commented 3 years ago

Should be fixed in v1.2.4.

vladar commented 3 years ago

Yeah, works just fine with 1.2.4. Thank you for the prompt fix!