npmd-client

[kesla]

Yo!

I have use case where I'd like to query npmd from another module.

AFAIK, there's no support for that currently in npmd, so I've so far manually connected to npmd with the help of multilevel.

Are you working on supporting npmd as a library currently? Would you like to have that functionality in npmd, or would it be better suitable as a library of it's own? Do you want me to work on getting this in npmd?

[dominictarr]

totally, doable - that is even how npmd works internally, what sort of stuff do you want to do?

yes! pr please!

[ralphtheninja]

Cool. Kesla for president! :D

[kesla]

Right now my usecase is that I'd like to iterate over all npm-module and reading some metadata from each. So doing a manual npmd-client that allowed med to do just that was really simple, but I'd of course like to do something proper.

Going forward I might want to do some other fun things with npmd, so stay tuned :smile:

[dominictarr]

@kesla so we should definately be able to pull something like that out, since it just uses multilevel anyway, we should discuss this in irc, as I have plans to rewrite stuff (but been busy)

[kesla]

I'm a bit busy these days, but let's discuss on irc later. I have another idea that I'd like to use npmd for that potentially will demand more from a npmd-client... And it's also a bit more badass.

[dominictarr]

well, we must definately discuss then! (I also have some future mad science planned...)

[kesla]

An update on this, I'm going to spend some time this weekend doing a POC of the idea I have for something where I'll be using npmd. I'll be doing it by creating a fork of npmd, rather than by fixing npmd. When that's done (and if it actually proves to work) we can discuss how to do it by extending npmd.

Code is king.

[dominictarr]

Awesome! I await in great anticipation of whatever it might be!

[dominictarr]

hey, how did you go with this?

[joaoafrmartins]

hey! im interested in this.. i was thinking of a multilevel sync to a level-fs instance using html5 file api with polyfills dependency installation in the browser. I was thinking something with a signature like lib(["sem@ver"], (dependencies) ->)... this shouldent be to hard to implement but i am wondering if any of u guys has beat me to it?

[dominictarr]

dropping the current npmd into the browser would likely be problematic, because you'd be likely to hit storage limits... although, you could probably take the same idea and make a striped down version that didn't replicate everything

what are you wanting to do? and what aspects of npmd are most appealing/important?

[joaoafrmartins]

im thinking a a rewrite of this where all dependencies are installed into indexdb... or rather some kind of sync to indexdb from a leveldb instance on a server... on just avoiding leveldb completly and install directly from github or npm... although this would mean the could needed to either be encrypted or opensourced... i am not thinking of a npm mirror like npmd but rather a package installation on indexdb... where some kind of manifest file determines wich dependencies are to be installed... and u just sync to be shure all versions of required packages are up to date... i guess it would make a lot of sense tu use something like git.js in this i just dont know whats the status of that project and was looking for someone to bounce ideas with.... but im thinking of a browser installation... btw if u need any code monkeys for cyphernet count me in!

[dominictarr]

oh, so you just want to be able to install modules into the browser?

have you seen browserify cnd? https://github.com/jesusabdullah/browserify-cdn

[dominictarr]

here is the service: http://wzrd.in/

[joaoafrmartins]

yes i've seen this but i havent tried it... this keeps a cache on the server wright? im thinking caching on the browser makes more sense... i guess i could just write a client that caches packages from this... probably makes more sense than rewritting the whole thing..

[joaoafrmartins]

i think the only real tradeoff is i would not be able to install private packages...

[joaoafrmartins]

unles som crypto magic... well i guess i hadnt really thought that throgh... thanks a lot... do u know any good html5 polyfills for the file api? have u had any experience with this https://github.com/Modernizr/Modernizr/wiki/HTML5-Cross-Browser-Polyfills#wiki-file-api? am i in for a world of hurt? do u think it is even worth it? thanks for all the good stuff btw!

[dominictarr]

I only know about this one: https://github.com/juliangruber/level-fs but I havn't used it.

what are you trying to use it in though, that needs installs? is this to deploy an app? I don't understand what you are actually trying to do, what do you want to use this in?

[joaoafrmartins]

yes.. like a browser installer.. the browser get a manifest and installs an app... the app is cached by the browser.. and everytime the url is visited the manifest is read to check for updates...

[joaoafrmartins]

thick clients for the browser... yes.. i've been looking throw this libraries... im new to leveldb... i think its awesome btw!

[joaoafrmartins]

im really sorry to bother u with this but please correct me if i am right... multilevel is a level client right?? the database lives in the server not the browser correct??? so files are not cached locally... they are cached in the server... if the files were to be cached locally.... and u would only diff the two manifest files... wouldnt that save a lot of bandwidth??

[dominictarr]

So, I think if this was gonna be effective, you'd want the loader thing to be very lightweight. I'd probably not bother to do a full install, but having a thing that updated js, and say, hashed it to verify it was correct, and something to detect that an update is necessary. I started on something like that over here: https://github.com/dominictarr/offlinerify

It uses a bootloader stored in appcache, and then loads a js bundle, and stores it in local storage.

[joaoafrmartins]

yes... this looks straight forward... awesome! so if the hash doesnt match u are throwing an error... i think it would make sense to treat that case as an update scenario do u agree?? i can write some code for that if like... i'll stop nagging u now i promise... ahaha awesome!

[dominictarr]

if the hash is wrong then it's an error. the bootloader requests manifest.json which contains the expected hashes of the files, then it requests those files, and checks them. Checking the hash is a security thing, if that fails it should error, or maybe try again later.

[dominictarr]

probably just print an error message, and continue using the old version.

[joaoafrmartins]

it looks like what i was looking for... havent had time to give it a spin yet but i will give u feedback when i do... i meant that about cyphernet... thank you tarr!! you are the boss!! ehehehe

[dominictarr]

@joaoafrmartins you should come and find me in #stackvm if you want to discuss.

Unfortunately, I don't quite know how to make use of a code monkey - If I had to herd code monkeys, then I can't imagine that being more productive than just writing the code my self. What I actually need are active collaborators - people who are interested in the area, want to experiment and learn about distributed systems, databases and crypto. Mad Scientists, not "code monkeys" - however, I am only too happy to offer advice and feedback - maybe like having Mad Grad Students.

I don't want to tell people what to do, what I want is to help self-directed individuals find their way.

[joaoafrmartins]

@dominictarr well i guess u would find that really hard... fisrt of all because i've got no uthers... second because i'm a bit o a pirate myself tarr! ahahah... anyway the part about mad grads and distributed, crypto and databases would be awesome... i will do that! and again thanks for all the nice stuff!!!

[dominictarr]

@joaoafrmartins you should come and hang out in #stackvm that is node.js mad science central.