search

jbenet / random-ideas

random ideas
juan.benet.ai
324 stars 12 forks source link

JRFC 19 - Let's Stop Installing Packages #19

Open jbenet opened 10 years ago

jbenet commented 10 years ago

Today's paradigm includes installing software. It's really silly, having to go find a particular package, and then download it manually. Our package managers should just make the code available. If it can be found in the registry, it should be importable in the code.

How?

Easy, mount the registry: 

/npm/<module-name>/<version>

Or, in my world:

/ipns/npmjs.org/<module-name>@<version>

Concerns

What about Security? is this safe!?

Security is not about installing software X at time Y, but about checking integrity (hash the code) and authenticity (sign the code). This could be done on import, every single time you run the code, which would be much safer than just hoping all your files are the same as when you last looked at them. You did look at all the modules you imported, right? You are sure that foobar module you used doesn't actually open a back door, right?

But what about production? Will I have to randomly download modules?

Not at all, things will be cached locally, and just make sure things stay local, why not pin them?

> mounted-npm pin module-i-care-about

Basically, make your "mounted registry" save things locally that you're going to use regularly. (IPFS will do this for you).

But, maybe I want to version lock?

And you should! Lock your local files to exactly the modules you want:

> echo bar@1.3 >> modules-i-want
> echo foo@1.0 >> modules-i-want
> cat modules-i-want | mounted-npm lock
> mounted-npm ls
bar@1.3
foo@1.0
jbenet commented 10 years ago

@groundwater you'll want this in NodeOS, i think :) -- give me two weeks and IPFS may be stable enough for it.

groundwater commented 10 years ago

This is kind of like a lazy install of modules.

I'm also interested in what things we can dream up when file-systems and mounts are easy and safe to hack on. Kinda like FUSE, but without the constant segfaults :smile:

jbenet commented 10 years ago

Kinda like FUSE, but without the constant segfaults

FUSE gives you segfaults!? I've yet to see this-- perhaps I'll run into it shortly. Is this osx or linux?

groundwater commented 10 years ago

It gives me segfaults because I'm causing them. Basically I'm a shitty FUSE developer =]

mlovci commented 10 years ago

http://xkcd.com/1367/

jbenet commented 10 years ago

@mlovci precisely. The divide between web and fs is really stupid and it's time to end it. We don't do http requests on ever file open, and with good reason. Need a better protocol :)

jbenet commented 10 years ago

@groundwater try fuse4js? shouldn't be able to segfault :)

groundwater commented 10 years ago

Retrieving and caching content is pretty straight forward. How would you deal with directory listing?

jbenet commented 10 years ago

@groundwater I wouldn't. Don't list root (-r+x on dir). Do list under keywords/, author/<username>/, etc.

groundwater commented 10 years ago

Would you use FUSE?

jbenet commented 10 years ago

More like, i will use FUSE ;)

groundwater commented 10 years ago

Too bad we can't just patch the fs module.

jbenet commented 10 years ago

For @mikolalysenko:

Below, it could easily be foo@version or foo/version. I'm opting for foo@version because i like it more. You can also keep the node_modules/ but i took it out below. Con

/ipns/npmjs.org/package/foo@1.0.0
/ipns/npmjs.org/package/foo@1.0.1
/ipns/npmjs.org/package/bar@1.0.0 
/ipns/npmjs.org/package/bar@1.0.0/foo@1.0.0  (same object as .../foo@1.0.0 above)
/ipns/npmjs.org/package/baz@1.0.0 
/ipns/npmjs.org/package/baz@1.0.0/foo@1.0.1 (same as .../foo@1.0.1)
/ipns/npmjs.org/package/bork@1.0.0/bar@1.0.0 (same as .../bar@1.0.0)
/ipns/npmjs.org/package/bork@1.0.0/baz@1.0.0 (same as .../baz@1.0.0)
jbenet commented 10 years ago

cc @mafintosh and @maxogden

mafintosh commented 10 years ago

@jbenet wouldn't I need to list the /ipns/npmjs.org/package (which can be HUGE) to match a semver using the above approach?

jbenet commented 10 years ago

@mafintosh where match here means resolve something like >=1.2.3 ? Ah, then maybe we do want name/version in top level:

/ipns/npmjs.org/package/foo/1.0.0
/ipns/npmjs.org/package/foo/1.0.1
/ipns/npmjs.org/package/bar/1.0.0/foo@>=1.0.0  # only needs to list /ipns/npmjs.org/package/foo/
/ipns/npmjs.org/package/baz/1.0.0/foo@^1.0.1
mafintosh commented 10 years ago

Yes this makes it a lot easier :) I would probably also not include sub dependencies. I don't really see the benefit of having them.

jbenet commented 10 years ago

@mafintosh we get subdependencies for free.

/ipns/npmjs.org/package/foo/1.0.0
/ipns/npmjs.org/package/bar/1.0.0                         
/ipns/npmjs.org/package/bar/1.0.0/foo@>=1.0.0
/ipns/npmjs.org/package/bork/1.0.0/bar@>=1.0.0  
# \o this resolves to bar/1.0.0 above, which already has foo@>=1.0.0

content-addressed deduplication :) !

mafintosh commented 10 years ago

Nice!!

jonathanmarvens commented 10 years ago

Random stranger stopping by to say that these are awesome thoughts :) .

@jbenet

- Jonathan

silky commented 9 years ago

check the nix package managed and nixos - http://nixos.org/ - https://nixos.org/nix/

BillDStrong commented 8 years ago

Why would this only be at that layer?

If you want to go that route, create a Linux distro that mounts IPFS early enough in the chain, then symlink all the normal filenames to IPNS addresses that always have the latest released version. Instant always up to date system.

burdges commented 8 years ago

You need reproducible builds to do this securely, along with Guix or NixOS tool for keeping package options straight.