kriszyp / lmdb-js

Simple, efficient, ultra-fast, scalable data store wrapper for LMDB
Other
479 stars 39 forks source link

bug(binary): `binary` store does not guarantee a `Buffer` on `get` #262

Closed dtfiedler closed 5 months ago

dtfiedler commented 7 months ago

Environment:

Example:

// index.mjs
import { open } from 'lmdb';

const lmdb = open({
    path: '/data/lmdb',
    encoding: 'binary',
});

const responseObject = await fetch('https://arweave.net/block/height/800079'); // large JSON object
const bigJsonObject = await responseObject.json();

await lmdb.put('my-buffer', Buffer.from(JSON.stringify(bigJsonObject), 'base64'));

const object = lmdb.get('my-buffer');
console.log(object instanceof Uint8Array); // true
console.log(object instanceof Buffer); // false
const isBuffer = Buffer.isBuffer(object); // false

Expectations:

We believe the reason is due to this line, where Buffers can be casted to Uint8Arrays under the hood.

As a workaround we explicitly check that we are returned a Buffer and if not, create one from the returned object:

const potentialBuffer = lmdb.get(key) 
if (potentialBuffer) {
  return Buffer.from(value.buffer);
}
kriszyp commented 7 months ago

This should be addressed in v2.9, I believe.