Level / memdown

In-memory abstract-leveldown store for Node.js and browsers.
MIT License
287 stars 37 forks source link

fails with level-ttl #66

Closed jfromaniello closed 5 years ago

jfromaniello commented 7 years ago

I am not sure if this is an issue with level-ttl or memdown, but I can't get this to work:

const level  = require('levelup');
const ttl = require('level-ttl');
const memdown = require('memdown');

var db = level({db: memdown});
db.put('foo', 'bar');
db.get('foo', (err, data) => console.log(data));
//prints bar

const aday = 24 * 60 * 60 * 1000;
var ttldb = ttl(db, { checkFrequency: aday });
ttldb.put('foo', 'bar', { ttl: aday })
ttldb.get('foo', (err, data) => console.log(data));
//prints undefined

It works if I use leveldown.

calvinmetcalf commented 7 years ago

what about running ttldb.get('foo', (err, data) => console.log(err, data));

jfromaniello commented 7 years ago

It is a NotFoundError , either through ttldb.get or with db.get

calvinmetcalf commented 7 years ago

hm likey an issue with ttldb , I can check into it

On Thu, Mar 16, 2017 at 9:54 AM José F. Romaniello notifications@github.com wrote:

It is a NotFoundError , either through ttldb.get or with db.get

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/Level/memdown/issues/66#issuecomment-287063603, or mute the thread https://github.com/notifications/unsubscribe-auth/ABE4n7b9YOns6wg1BmEi2L4m9Aeqmdswks5rmT8vgaJpZM4MeZJx .

calvinmetcalf commented 7 years ago

wait I know ttldb.put('foo', 'bar', { ttl: aday }) you're not waiting for the callback here, I'm frankly surprised it works in leveldown.

yonjah commented 7 years ago

@calvinmetcalf you should ignore the minor issues in the example code and see that this is an actual bug in memdown/level-ttl integration.

the issue is that memdown is using ltgt compare which assumes all values are from the same type and will produce inconsistent result when with mixed values of buffers and strings and since level-ttl uses Buffer keys it will break if other code uses string keys ( I assume leveldown doesn't care about this mixed value issues since it probably converts all keys to buffers when doing the js to cpp shift but I haven't dug into it too much)

@jfromaniello as a work around you can convert your keys to Buffers the following code seems to work -

const ttldb = ttl(db, { checkFrequency: aday });
const key = Buffer.from('foo');
ttldb.put(key, 'bar', { ttl: aday }, (err) => {
    err && console.log(err);
    ttldb.get(key, (err, data) => console.log(data));
})
jfromaniello commented 6 years ago

@yonjah Thank you very much and sorry for the delay on my response... I missed the notification for your response.

By doing this the TTL module seems to work, but I break another part of my code where I was doing something like this:

    const readStream = type.db.createReadStream({
      gte: prefix,
      lte: `${prefix}~`,
    });

If I do this I don't get any result, I tried converting these to Buffers, but I don't get any result back.. Any ideas how can I get that working?

yonjah commented 6 years ago

@jfromaniello sorry I don't have a lot of experience with levelDB. I was only looking at this issue since I was considering using limitd. I can try diving back into it when I'll have some spare time but I'm not sure when it will be

vweevers commented 5 years ago

This is likely https://github.com/Level/level-ttl/issues/68 or more specifically https://github.com/Level/level-ttl/issues/68#issuecomment-480505112.