Investigate level-js status

dignifiedquire commented 7 years ago

The current master of https://github.com/maxogden/level.js is not up to date with the rest of the level eco system. Currently we are depending on https://github.com/timkuijsten/level.js/tree/idbunwrapper which is near complete rewrite supporting use cases like empty values for us.

We should try and see in https://github.com/maxogden/level.js/issues/46 what it takes to finish this and get it released or start maintaining our own version.

daviddias commented 7 years ago

@pgte you are very close to the Level ecosystem. Any comment here?

pgte commented 7 years ago

I'm less familiar with the level-js than the node side of it.. Anyway, I've been implementing a level down store myself, and the abstract level down test suite is impressive, so maintaining such a thing is not something to be taken light-heartedly, specially if this is touching the (de)serialising / (de)encoding parts.. :) Which leads me to asking: What problem are we trying to prevent / solve here?

dignifiedquire commented 7 years ago

The issue is that the original level.js is not usable for us which is why we are currently using a fork, which brings with it all the issues of depending on a fork which is not maintained nor officially released.

Main issues with level.js master were the iterators not being usable as far as I remember and some other thing that are fixed in the fork

daviddias commented 7 years ago

@dignifiedquire is there a PR to the main one?

dignifiedquire commented 7 years ago

@diasdavid no just this comment:https://github.com/maxogden/level.js/issues/46#issuecomment-182144244

ralphtheninja commented 7 years ago

@dignifiedquire o/ I have begun working on this.

ralphtheninja commented 7 years ago

Quick and dirty roadmap:

move level.js to level org, me, @juliangruber and @vweevers are active there now, I have asked @maxogden to move it and he's cool with it
spend some time on fixing most obvious problems to get it up to par with abstract-leveldown
replace the idb-wrapper
get snapshots working
other issues/unknown problems?

I haven't looked at the fork just yet, just focusing on the big brush for now :)

dignifiedquire commented 7 years ago

@ralphtheninja great to hear, the fork works very well and is also a good deal faster than the official version if I remember correctly. I also think the other reason to use the fork were issues with the iterators, but I don't remember the details atm.

juliangruber commented 7 years ago

I'm happy with what @ralphtheninja suggests!

ralphtheninja commented 7 years ago

@ralphtheninja great to hear, the fork works very well and is also a good deal faster than the official version if I remember correctly. I also think the other reason to use the fork were issues with the iterators, but I don't remember the details atm.

Cool. The thing with iterators right now I think is that it doesn't terminate properly, it keeps going on under the hood but ignores calling back with data so it's slow.

Actually, maybe it's easier to take the fork and start working on that to get it up to the latest abstract-leveldown.

richardschneider commented 6 years ago

Why are we dependent on level-js and level-down? This should all be handled by datastore-level.

Also, datastore-level uses "level-js": "^2.2.4" NOT "level-js": "timkuijsten/level.js#idbunwrapper"

ralphtheninja commented 6 years ago

Unfortunately I don't have time for this atm because life. That said, I haven't forgotten about this. Maybe @vweevers can take this up or maybe I'll jump on it later on, but right now it's not possible :cry:

vweevers commented 6 years ago

I'm currently working on memdown and abstract-leveldown (to fix weirdness surrounding serialization, which also helps to clean up level.js). After that I can put some time into level.js - but only on weekends atm, so it will take a while.

Regarding:

The thing with iterators right now I think is that it doesn't terminate properly, it keeps going on under the hood but ignores calling back with data so it's slow.

The fork will by default keep reading to fulfill the snapshot guarantee dictated by abstract-leveldown. If users want better performance, they can opt-in to backpressure, losing snapshot guarantees. I think this approach works well (and I don't see any other option), so will probably copy it.

vweevers commented 6 years ago

Hey everybody! Good news: v3.0.0-rc1 is out and can be installed with npm i level-js@next.

achingbrain commented 4 years ago

This issue is quite old - level-js is healthy & well maintained, I think it can be closed now.

ipfs / js-ipfs-repo

Investigate level-js status #127