"Key not found in database" from db.get, when fetching keys that are known to exist

[pgleghorn]

HI, I'm trying to read a leveldb file which contains some existing keys & values, I can successfully fetch all keys/values using createKeyStream etc, but I fail to fetch some specific keys that I know exist.

For example using createReadStream (as in https://www.npmjs.com/package/level#createReadStream) to list all keys/values, I see all expected keys/values:

_file://dbInitContext = {"0000_initial":"finished","0001_add_archived_resources_table":"finished",[...snip]
_file://migrationFinished = true
_file://settings = {"lineWrapping":true,"previewType":"parsed","trimKeysAndValues":false,"agent-setting":null,[...snip]
VERSION = 1

I can successfully fetch the "VERSION" key/value eg using db.get('VERSION'), but fail to fetch the others.

Example code:

const level = require('level')
async function test() {
  const db = level('leveldb')
  console.log(`fetching VERSION: ${await db.get('VERSION')}`);
  console.log(`fetching dbInitContext: ${await db.get('_file://dbInitContext')}`);
}
test();

Output:

$ node index.js
fetching VERSION: 1
(node:53915) UnhandledPromiseRejectionWarning: NotFoundError: Key not found in database [_file://dbInitContext]
    at /Users/a8409107/git/leveldb-test/node_modules/levelup/lib/levelup.js:176:15
    at db.get (/Users/a8409107/git/leveldb-test/node_modules/encoding-down/index.js:75:21)
(node:53915) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:53915) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I'm using level@7.0.0 with node v10.17.0. The particular leveldb I'm querying isn't something of my own creation, it's part of Postman API utility (an electron application, using leveldb for local storage).

Thx Phil

[vweevers]

For example using createReadStream

Can you share that code as well? So we can see the difference.

[pgleghorn]

Hi @vweevers , sure the createReadStream is pretty much a copy from the readme:

function fetchAll() {
  const db = level('leveldb')
  db.createReadStream()
    .on('data', function (data) {
      console.log(data.key, '=', data.value)
    })
    .on('error', function (err) {
      console.log('Oh my!', err)
    })
    .on('close', function () {
      console.log('Stream closed')
    })
    .on('end', function () {
      console.log('Stream ended')
    })
}

But I just figured out the problem, there are some non-printable characters in the keys. I used createReadStream to fetch everything and write the keys into files, so that I could hex dump them:

$ xxd key3
00000000: 5f66 696c 653a 2f2f 0001 6462 496e 6974  _file://..dbInit
00000010: 436f 6e74 6578 74                        Context

No idea what's up with these funky hidden chars, but now I succeed to fetch them via db.get using a suitable matching key:

  var funkyKey = "_file://\000\001dbInitContext"
  console.log(`fetching dbInitContext with a funky key: ${await db.get(funkyKey)}`);

n.b I thought I might need to use {keyEncoding: 'binary'} on the db.get options, but turns out it works regardless.

Thx Phil