eugeneware / subindex

Generic pluggable indexing system for leveldb/levelup. Designed to be used with the level-queryengine query engine.
18 stars 6 forks source link

Sample code not working #8

Open peoro opened 8 years ago

peoro commented 8 years ago

I tried to run the sample code:

var levelQuery = require('level-queryengine'),
    jsonqueryEngine = require('jsonquery-engine'),
    levelup = require('levelup'),
    db = levelQuery(levelup('/tmp/my-db', { valueEncoding: 'json' }));

db.query.use(jsonqueryEngine());

db.ensureIndex('num');
db.ensureIndex('tags');

db.put('hey', {tags: 'tag1', num:1}, function (err) {
    // db.query({ $and: [ { tags: 'tag1' }, { num: { $lt: 100 } } ] })
    db.query({ tags: 'tag1' })
        .on('data', console.log)
        .on('stats', console.log);
});

in an empty directory (just after a npm install level-queryengine jsonquery-engine levelup leveldown), and it fails:

buffer.js:739
        return this.hexWrite(string, offset, length);
                    ^

TypeError: Invalid hex string
    at TypeError (native)
    at Buffer.write (buffer.js:739:21)
    at fromString (buffer.js:198:26)
    at Function.Buffer.from (buffer.js:113:12)
    at new Buffer (buffer.js:94:17)
    at decode (/tmp/fff/node_modules/subindex/index.js:14:26)
    at DestroyableTransform._transform (/tmp/fff/node_modules/subindex/index.js:55:27)
    at DestroyableTransform.Transform._read (/tmp/fff/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)
    at DestroyableTransform.Transform._write (/tmp/fff/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:172:12)
    at doWrite (/tmp/fff/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:237:10)

I tried to debug a little, and it looks like db.indexDb.createReadStream returns keys containing the level-sublevel metadata (e.g. ÿindexesÿa0707461677300707461673100706865790000).

I'm not really sure why this is happening. Am I doing anything wrong, or did anything broke?

peoro commented 8 years ago

After some more digging I found out that the latest level-queryengine, 3.0.1, depends on subindex version 2.0.0, which, in turn, depends on level-sublevel 5.2.0.

It looks like level-sublevel 5 has the issue I mentioned above, while 6 fixes it: the read stream ignores sublevels. Some scripts I found and some level-sublevel GitHub issues seem to confirm my hypothesis.

I believe that the latest version of subindex (depending on level-sublevel 6) doesn't have any issue, and the problem here is level-queryengine, which should depend on the latest subindex.

I wonder how level-queryengine could ever work: all the mentioned projects depend on specific (well, caret-preceded...) versions of each other...