npm on IPFS - Githubissues

jbenet commented 9 years ago

[x] ipfs-blob-store
[x] ipfs-daemon-ctrl control whatever daemon is installed
[x] plug in ipfs-blob-store to reginabox
[ ] figure if we can 302 publish
[x] mirror the registry
[x] bundle it all into one module
[x] make physical nodes

cc @bengl

whyrusleeping commented 9 years ago

just idle thought? Or discussion?

jbenet commented 9 years ago

@whyrusleeping notes from discussion in person :)

bengl commented 9 years ago

As mentioned in the first checkbox, one blocker for this is that someone needs to make a ipfs-blob-store adhering to abstract-blob-store. I'm working on this on and off in my spare time. Slowly. If someone beats me to it, yay!

seidtgeist commented 9 years ago

@jbenet Hmm... After you explained the ipfs docker mount hack yesterday, what if we mount ipfs on node_modules and require('hash')? Why am I even asking and not just trying this? brb

whyrusleeping commented 9 years ago

@ehd how did that work out?

seidtgeist commented 9 years ago

@whyrusleeping It works :)

$ cat add.js
'use strict';

module.exports = function(a, b) {
  return a + b;
};

$ ipfs add add.js
added QmXctmnj6tEMSJsb433jKwcdGwuBy4uGMnqva5Nkvp5E7x add.js

$ cat prog.js
'use strict';

var add = require('QmXctmnj6tEMSJsb433jKwcdGwuBy4uGMnqva5Nkvp5E7x');

console.log(add(1, 2));

$ node prog.js
3

Two things:

I'd rather not rely on the filesystem or fuse, that's doable
I'd rather require names instead of SHAs and move the mapping to metadata

whyrusleeping commented 9 years ago

@ehd thats great! what would the ideal interface be for you to want to use ipfs for js modules?

seidtgeist commented 9 years ago

Ideally we'd have a simple bootstrapping js/node program in ipfs that can load other things from ipfs using require. Reminds me of the StackStream opcodes actually ☺️ Anyway, without a file system the ideal delivery mechanism would be the true ipfs network, so baking a simple ipfs client into a runnable JavaScript bundle sounds like an idea worth trying. Initially this could be done by just using the global or local ipfs http gateway for loading code. I'm not sure whether a dynamic module loader (require on demand) or a dynamic bundler (compile program, then start program) would be more viable. Whatever runs in <script src="browser-fetchable-address-on-ipfs"/> or vanilla node I guess.

Another related idea we've bounced: storing computational results in ipfs. Say you have blobs for js-compile and js-source-with-links; you can load both and run js-compile(js-source-with-links), then store the result and use that as the above entry point. It's basically how all browser side JS already works, just that it's very sad how we store build results in dumb files without linking to how they've been conceived (creation metadata! cc @jbenet).

Side note: Ironically I'm in a place without wifi and just looked up stackstream on my mac's browser history, but of course I can't access the already cached page. Idea: any ipfs browser/browser extension should pin history items. Immediate benefit over any other real world browser.

jbenet commented 9 years ago

Update on the list above: @krl made https://github.com/ipfs/node-ipfsd-ctl

@ehd all of that sounds awesome!

Ideally we'd have a simple bootstrapping js/node program in ipfs that can load other things from ipfs using require.

Yeah! like a special require that uses ipfs directly. Could patch npm to make requires that start with "/ipfs/..." require straight out of ipfs.

Btw, what @bengl and I want to do above with reginabox will work with stock npm and require.

Reminds me of the StackStream opcodes actually ☺️ Anyway, without a file system the ideal delivery mechanism would be the true ipfs network, so baking a simple ipfs client into a runnable JavaScript bundle sounds like an idea worth trying.

that sounds awesome

Initially this could be done by just using the global or local ipfs http gateway for loading code. I'm not sure whether a dynamic module loader (require on demand) or a dynamic bundler (compile program, then start program) would be more viable. Whatever runs in <script src="browser-fetchable-address-on-ipfs"/> or vanilla node I guess.

@krl's https://github.com/ipfs/node-ipfsd-ctl can make ephemeral ipfs nodes using the go-ipfs package, so you dont even have to install ipfs beforehand. it's huge (cause go-ipfs is huge) but once we have node-ipfs this will be a drop-in replacement.

Another related idea we've bounced: storing computational results in ipfs. Say you have blobs for js-compile and js-source-with-links; you can load both and run js-compile(js-source-with-links), then store the result and use that as the above entry point. It's basically how all browser side JS already works, just that it's very sad how we store build results in dumb files without linking to how they've been conceived (creation metadata! cc @jbenet).

Absolutely! And-- i want to make a way to do browserify with partial results separated out. THOUGH, rabin fingerprinting will get us 80% of the way there without the worries anyway.

Side note: Ironically I'm in a place without wifi and just looked up stackstream on my mac's browser history, but of course I can't access the already cached page. Idea: any ipfs browser/browser extension should pin history items. Immediate benefit over any other real world browser.

Yeahhh i really want this. wonder what the overhead + pains of doing this might be... hard to get right because people dont hash things or use cache headers correctly :/

bengl commented 9 years ago

Could patch npm to make requires that start with "/ipfs/..." require straight out of ipfs.

You'd have to patch node. All the require logic is self-contained in node, not npm.

There are also many lifecycle scripts and compilation steps and all kinds of other things that are done to modules during the install process. In some modules, the results of these scripts are different depending on where they are in the node_modules tree.

I think it's much more practical to keep a strict separation between the source of packages (ipfs/ipns, in an ideal world) and the current install directory. Requiring them directly might work for JS-only modules with no lifecycle scripts. But then you're not supporting a huge chunk of available modules.

I'm only really talking about node here, but for people using frontend build pipelines that depend on npm, the same problems can arise.

So yeah, I'm not sure grabbing node dependencies directly from ipfs at run-time is realistic.

jbenet commented 9 years ago

@bengl ah thanks! makes a lot of sense.

i guess we could do it to specific packages people decide to do this with (i.e. people explicitly publish them + require them with hashes, so they take on the responsibility of not making all the lifecycle stuff happen?) not sure.

bengl commented 9 years ago

Yeah for sure. ipfs-pm/ipfspm/ippm. You'd pretty much have to make a clear separation from npm at that point, I'd think.

I still would like to see the ipfs leveraged as the package source, as we discussed before. Should be easy once ipfs-blob-store exists :)

jbenet commented 9 years ago

@bengl maybe we can hack that (ipfs-blob-store) out this week. I'm in bay area atm. (cc @krl @travisperson @mappum @diasdavid in case they're inspired)

travisperson commented 9 years ago

Definitely, I took a look at the spec today. @bengl do you have any of the work you've made towards it published anywhere?

bengl commented 9 years ago

So really we're looking at potentially two different use-cases for ipfs-blob-store. Might be better off as two different modules. I'm not sure.

Use Case 1: Raw Blob Store

In this case, we're really just plopping blobs onto IPFS and getting their IPFS paths back in the metadata for later retrieval. So in createWriteStream, you wouldn't provide a key in the opts as that's what you'd be getting back from IPFS (i.e. the IPFS path). In createReadStream, the key you'd pass in would be the IPFS path, and it would stream back the blob at that path.

Use Case 2: IPNS Mutable File System Thing

In this case (which could use the previous case under the hood), in the constructor for the blob store, we'd provide all the necessary data to read and write to an IPNS namespace (keypair? just the hash? Sorry, my IPNS knowledge is rather limited). Here, createWriteStream would use the key option from the args to determine where to put the blob, and the metadata returned would be kind of irrelevant, but would have the same key. Behind the scenes it would update IPNS properly. So the key would be something like /path/in/namepsace. Similarly for createReadStream.

Unfortunately I don't spend as much time on this as I'd like, so I'm unfamiliar with how things actually work in IPNS-land right now. Sorry!

Here's the abstract-blob-store API reference.

jbenet commented 9 years ago

@bengl yep, you've got it right :)

We'd want a third module: unixfs-blob-store which would read an ipfs object as unixfs and stream out the full file (or compute StackStream)

seidtgeist commented 9 years ago

Pardon my late replies, I’m on a vacation until Thursday.

You’d have to patch node. All the require logic is self-contained in node, not npm.

That’s true. But you can change the meaning of require through compilation, e.g. like webpack does. The JavaScript bootloader could do this when loading modules from ipfs.

But then you’re not supporting a huge chunk of available modules.

That’s unfortunately true, but I think we can do lots of useful stuff without modules that rely on native compilation. Most compilation I see when running npm install is (dev)dependencies installing native file watchers. If native dependencies are a must, there could be pre-compiled binaries per runtime that could be grabbed from ipfs. Most backend or hybrid JS code I’ve written runs in both node and browsers and doesn’t depend on native dependencies. I could do lots of interesting stuff without them. Maybe @grncdr can also comment on that.

travisperson commented 9 years ago

First stab at ipfs-blob-store

https://github.com/ipfs/ipfs-blob-store

Currently uses raw blocks, and is only content addressable. It does not use ipns at the moment, passes 45 of the tests (out of a bunch more, I don't fully understand tape).

jbenet commented 9 years ago

i think it's time to resurrect this effort.

cc @mappum @daviddias @richardlitt

whyrusleeping commented 9 years ago

npm uses tar right? ipfs tar is so ready.

daviddias commented 9 years ago

:+1:

bengl commented 9 years ago

:+1: :boom:

daviddias commented 9 years ago

Looking for some clarifications:

Do we need https://github.com/yahoo/reginabox for this? Or should I be just looking at https://github.com/davglass/registry-static , as it already creates npm endpoints
Is registry-static ready to handle publishes? (not fully getting it from docs)
Is there a way to proactively cache the whole npm built into registry-static?

this is so awesome!

bengl commented 9 years ago

Do we need https://github.com/yahoo/reginabox for this? Or should I be just looking at https://github.com/davglass/registry-static , as it already creates npm endpoints

Just registry-static. I maintain it as part of my day job, so LMK if you need anything there. reginabox is just a fancily-pre-packaged version of registry-static.

Is registry-static ready to handle publishes? (not fully getting it from docs)

No. registry-static is not actually equivalent to an npm registry. It's only useful for mirroring from the public registry (or any other registry that supports couchdb change feeds). For a full-on registry replacement for internal or alternative usage, you need to either run your own couchdb, or something like cnpm.

Is there a way to proactively cache the whole npm built into registry-static?

Yes, that's the first thing registry-static does when you start it up (but it takes like 2 days).

hackergrrl commented 9 years ago

Hi. Trying to make sure I grok the big picture here, in terms of what an ipfs-npm request looks like. Please correct me as I go. ;)

User installs the ipfs-npm command, or whatever we call it
User runs ipfs-npm foobar
I'm not clear on how look-up happens, to map a module name to an IPFS address. One option is to use IPNS, and claim that "all ipfs-npm modules are published to this pubkey. This means all clients know that to get foobar the path will always be /ipns/<static pubkey we control>/foobar. Do we want that kind of control? It would mean centralizing the publish operation.
Easy part: we grab the module tarball from IPFS and install.

How does ipfs-blob-store fit in? Is this what reginabox uses to retrieve the blobs?

hackergrrl commented 9 years ago

@whyrusleeping helped me better understand the intent here. Looks like I was getting ahead of myself; it sounds like the first goal is to opt for decentralized before distributed: let users run their own npm registry that uses IPFS as the blob store rather than the local fs.

Lingering Q: how will our reginabox map module names to IPFS addresses?

whyrusleeping commented 9 years ago

(decentralized first, then distributed: https://ipfs.io/ipfs/QmZpfhN5rucQC4kx7Gu5udS8FXxsxMddiDCvG8WFjG8SMv)

daviddias commented 9 years ago

Just PR'ed ipfs-blob-store using mfs (mutable file system) https://github.com/ipfs/ipfs-blob-store/pull/5

@bengl I'm being unable to attach a custom blob-store to registry-static, looking at the --help menu, the --blobstore option doesn't appear (for ref posted the help menu below) and when I run something like » registry-static -d npm.ipfs.io -o /npm-ipfs --blobstore=registry-ipfs-blob-store.js, I get this error:

module.js:338
    throw err;
    ^

Error: Cannot find module 'registry-ipfs-blob-store.js'
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:286:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/usr/local/lib/node_modules/registry-static/lib/args.js:108:25)
    at Module._compile (module.js:434:26)
    at Object.Module._extensions..js (module.js:452:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)

Although I have the module exporting the blob store:

$ ls -al                                                                                                                                                                                                                                                                         
total 464
# ...
-rw-r--r--   1 david  staff      84 Oct 12 19:36 registry-ipfs-blob-store.js
# ...

$ cat registry-ipfs-blob-store.js
var ipfsBlobStore = require('ipfs-blob-store')
module.exports = ipfsBlobStore().mfs

Could you give me a hint on how to do it? Thank you :)

note: `mfs` is only available on IPFS dev0.4.0

$ registry-static --help
registry-static@2.0.0

Examples:
  registry-static -d my.registry.com -o /var/www/registry
  registry-static -c /path/to/myconfig.json

Options:
  --help, -h    Show help
  --config      Use a config file
  --domain, -d  The domain replacer                                 [required]
  --dir, -o     The output directory                                [required]
  --registry    The registry to mirror from                                     [default: "http://registry.npmjs.org/"]
  --limit, -l   Limit the number of concurrent downloads                        [default: 10]
  --user, -u    Set the user that this process should change to
  --group, -g   Set the group that this process should change to
  --hooks       Provide a hooks module.
  --clean       Clear the sequance file
  --check       Crawl the tarballs and check all of their shasums
  --sync        Crawl the json files and look for outdated index files
  --report      Used with --check, write a json report for all missing files
  --log         The file to log the output to
  --quiet       Turn down the logger
  --index       An index.json for the whole registry                            [default: "/usr/local/lib/node_modules/registry-static/defaults/index.json"]
  --error       A 404 file                                                      [default: "/usr/local/lib/node_modules/registry-static/defaults/404.json"]
  --replport    Port to listen on for REPL
  --one         Sync only the provided package, right now, then quit
  --spawn, -s                                                                   [default: 20]
  --tmp, -t                                                                     [default: "/var/folders/_y/h419_yvx6tb6wjkgjr6gp0p40000gn/T"]

bengl commented 9 years ago

@diasdavid The value passed in via --blobstore=<...> is just passed into require, and isn't relative to your local directory, so your invocation is going to have to look like:

registry-static -d npm.ipfs.io -o /npm-ipfs --blobstore=/full/path/to/registry-ipfs-blob-store.js

Sorry about that :disappointed: . I need to add either better docs or support for relative paths for that option.

daviddias commented 9 years ago

wooot! it is going

2015-10-13T16:49:46.584Z static [info] [116] downloading http://registry.npmjs.org/Reston/-/Reston-0.1.1.tgz
2015-10-13T16:49:46.584Z static [info] [111] downloading http://registry.npmjs.org/Reston/-/Reston-0.2.0.tgz
2015-10-13T16:49:46.882Z static [info] [116] (200) /Reston/-/Reston-0.1.1.tgz is 6 kB
2015-10-13T16:49:46.883Z static [info] [116] finished downloading http://registry.npmjs.org/Reston/-/Reston-0.1.1.tgz in 0.299 seconds
2015-10-13T16:49:46.884Z static [info] [111] (200) /Reston/-/Reston-0.2.0.tgz is 6.8 kB
2015-10-13T16:49:46.885Z static [info] [111] finished downloading http://registry.npmjs.org/Reston/-/Reston-0.2.0.tgz in 0.301 seconds
2015-10-13T16:49:46.886Z static [info] [117] downloading http://registry.npmjs.org/Reston/-/Reston-0.1.1.tgz
2015-10-13T16:49:46.887Z static [info] [112] downloading http://registry.npmjs.org/Reston/-/Reston-0.2.0.tgz

from the registry-static README

WARNING: This may take quite a while to complete and a large amount of disk space (more than 283GB at last check)

@whyrusleeping you were saying that you had a gigabit link ? :P I need to find some disk space

daviddias commented 9 years ago

@bengl is there a way to run registry-static without downloading the npm registry? We are looking into getting a big node in the network with all of NPM and then the user nodes would only download the modules they need using IPFS.

bengl commented 9 years ago

@diasdavid Downloading the npm registry is literally the whole point of registry-static. Downloading the modules one needs is the role of the npm client.

If you're getting a big node in the network with all of npm, all someone needs to do is configure a gateway server to serve up index.json as a directory listing (in much the same way index.html works on gateway.ipfs.io) and then that gateway can be used by npm clients for installs.

daviddias commented 9 years ago

@bengl what I'm thinking is that each user has it's own registry-static with a empty repo that when asks IPFS through ipfs-blob-store to 'createReadStream(module name)', that module comes from the IPFS network and since we have one very fat node with all of the modules, all of them will be available. Once some peers start downloading some of the modules, then these peers can also serve the modules to other peers

bengl commented 9 years ago

@diasdavid so each user in your ideal setup doesn't need to be running registry-static, as that's taken care of by your fat node. They just need ipfs and a little bit of glue.

What each user then needs for glue is either a very thin web server to run locally that (mostly) just proxies back to ipfs, or a customized npm client that talks to ipfs directly. If you make a thin web server, it will have to modify metadata so that it points to tarballs in the correct location, but this is not hard.

This way you get all of this as well:

That module comes from the IPFS network and since we have one very fat node with all of the modules, all of them will be available. Once some peers start downloading some of the modules, then these peers can also serve the modules to other peers

daviddias commented 9 years ago

Important piece of information from IRC discussion

09:23 <bengl> daviddias: it sounds like the 'local registry-static per user' your're talking about is equivalent to the 'thin web server' i'm describing on github
09:23 <+daviddias> bengl: reading from your comment of your issue, I believe we are in the same webpage
09:23 <+daviddias> ahah I meant, 'same page'
09:24 <+daviddias> yes, exactly :) 
09:24 <bengl> hahaha yeah but it made for a good pun given the context
09:24 <ogd> lol
09:25 <+daviddias> bengl: wouldn't it be more easier to have a option to skip the step of downloading everything in registry-static and use that code?
09:25 <bengl> daviddias: no, because there is no other code, as that's all it does
09:26 <bengl> reginabox (which i made as a way to get people up and running with registry-static quickly) includes a thin web server like the one i'm describing
09:27 <bengl> so feel free to steal it from there :)
09:27 <bengl> registry-static does not include a web server
09:27 <bengl> because it's intended to be used with something like ngninx
09:32 <+daviddias> oh, so I missed to understand that
09:35 <+daviddias> awesome, thanks bengl  :)

whyrusleeping commented 9 years ago

~~@bengl I think what @diasdavid was wanting was a way to do lazy loaded modules.~~

Whoops, github didnt update before i replied...

jbenet commented 9 years ago

this all sounds good to me. we'll have a box for the fat node soon

seidtgeist commented 9 years ago

Had an idea, been wondering whether that's being tried or pursued:

I'd like to have a PEER that offers a FILE of PACKAGE at VERSION as such:

/ipns/PEER/npm/PACKAGE/VERSION/path/to/FILE

Would something like this make sense? In a way, it would be useful to have a good way to access npm packages canonically, and have a trusted authority for feelgood-immutable npm@version names to ipfs objects.

jbenet commented 9 years ago

Agreed. In time this could be a key npm itself controls.

I wouldn't count too much on specific IPNS names yet until we have easy ways of delegating record signing through auth agents, so we can keep important keys more secure and isolated from the Internet.

— Sent from Mailbox

On Tue, Nov 3, 2015 at 9:55 AM, Stephan Seidt notifications@github.com wrote:

Had an idea, been wondering whether that's being tried or pursued: I'd like to have a PEER that offers a FILE of PACKAGE at VERSION as such:
/ipns/PEER/npm/PACKAGE/VERSION/path/to/FILE
Would something like this make sense? In a way, it would be useful to have a good way to access npm packages canonically, and have a trusted authority for feelgood-immutable npm@version names to ipfs objects.

Reply to this email directly or view it on GitHub: https://github.com/ipfs/notes/issues/2#issuecomment-153378575

jbenet commented 9 years ago

(The root ipns name would point to another ipns key, the current signing key from a rotation, etc. that way can revoke it by moving ipns -> ipns ptr at a given time, bounded only by record lifetimes)

— Sent from Mailbox

On Tue, Nov 3, 2015 at 9:55 AM, Stephan Seidt notifications@github.com wrote:

Had an idea, been wondering whether that's being tried or pursued: I'd like to have a PEER that offers a FILE of PACKAGE at VERSION as such:
/ipns/PEER/npm/PACKAGE/VERSION/path/to/FILE
Would something like this make sense? In a way, it would be useful to have a good way to access npm packages canonically, and have a trusted authority for feelgood-immutable npm@version names to ipfs objects.

Reply to this email directly or view it on GitHub: https://github.com/ipfs/notes/issues/2#issuecomment-153378575

daviddias commented 9 years ago

@ehd that is similar to what we are doing, where we have one PEER that has the entire npm by a similar folder structure (the one used in registry-static) and then the others, by copying that root path /npm have access to the folder structure and respective hashes, which then can be server by 1 or more nodes.

seidtgeist commented 9 years ago

I wouldn't count too much on specific IPNS names yet until we have easy ways of delegating record signing through auth agents, so we can keep important keys more secure and isolated from the Internet.

(The root ipns name would point to another ipns key, the current signing key from a rotation, etc. that way can revoke it by moving ipns -> ipns ptr at a given time, bounded only by record lifetimes)

Sounds great. Can't wait.

@ehd that is similar to what we are doing, where we have one PEER that has the entire npm by a similar folder structure (the one used in registry-static) and then the others, by copying that root path /npm have access to the folder structure and respective hashes, which then can be server by 1 or more nodes.

What do you mean by "the others"?

Here's an interesting project by @mjackson: http://npmcdn.com – It's a heroku app with a CDN in front, serving files out of npm packages.

Idea: Use trusted computation instead of trusted names for this problem (Distributed request network?)

f = hash of a javascript function (could be any lang with sufficient metadata or linking (unikernel!)) x = hash of an input

Someone else executes it for me, caches the result. I trust them to provide y = f x

If I paid them to do this for me, and they provided a binding legal contract: Very nice and trustworthy. You know who's got at providing a useful and stable environment for running code? CI services. There probably is a nicer blockchainy/ethereum smart contract way, but I'm not well versed here.

Something like npmcdn would be trivial with this: f is the lookup function, x is {name, version, pathToFile}, y is the computation result.

/ipcompute/funhash/inputhash -> result?

There are many things we could use that work like npmcdn

Benefit:

Anyone could do the computation
Code, inputs and outputs hosted on IPFS
Reproducible/verifiable
Fault-tolerant (have multiple nodes computing & caching)

Q:

Is this just something on top of ipns? pub-sub? req-res? notifications?

daviddias commented 9 years ago

@ehd like Amazon lambda/WebTask for the network? That is a field that is very very exciting and something we keep going back, it is definitely a valid idea.

Now when it comes to npm on IPFS, what we want is to be able to install the modules from the network, which is far less ambitious that having a universal way to do computation in a P2P network (btw I did work towards that goal in my MSc thing that I got to share at DTN last May, we can talk more or open a issue about it :):)).

The main challenge was, without forcing users to adopt a new way to install their modules, how could we offer the same npm install module-name experience? Well, the answer to that was IPNS and mfs.

Quick update on state of things

[x] got reginabox to use a blob-store driver for storage https://github.com/diasdavid/reginabox
[x] read only mode (avoid forcing the download of all the modules, useful if ipfs-blob-store is being used and the modules are already in the network)
[x] ipfs-blob-store
[x] fat storage node ready to glob the entire npm
[ ] rebase ipfs 0.3.9 onto 0.4.0 so that the new js-ipfs-api works again (which is a dependency of ipfs-blob-store)

seidtgeist commented 9 years ago

@ehd like Amazon lambda/WebTask for the network? That is a field that is very very exciting and something we keep going back, it is definitely a valid idea.

Yeah, it's so cool! I'll try to visualize my idea on the weekend.

The main challenge was, without forcing users to adopt a new way to install their modules, how could we offer the same npm install module-name experience? Well, the answer to that was IPNS and mfs.

And that's awesome. It'll be really cool and useful to have the that for npm end users and also the idea above. It relies on some system serving all the archives and metadata, and using npm on ipfs instead of the registry brings those benefits with it.

daviddias commented 9 years ago

The moment is near, you hear! :D

So, right now, using IPFS 0.4.0, with mfs, we can, using the new shiny js-ipfs-api and ipfs-blob-store, put the npm registry on IPFS and serve modules from there. To make this happen (and please replicate it so it works for everyone).

git clone git@github.com:diasdavid/reginabox.git
cd reginabox && npm i && npm link
cd .. && mkdir custom-blob-store
Create a index.js file with module.exports = require('ipfs-blob-store')().mfs && npm i ipfs-blob-store
Install IPFS dev0.4.0 branch and ipfs daemon
run reginabox mirror npm-registry <full-path-to-custom-blob-store>/index.js

This is to download the entire npm into an IPFS Node, once our fat Node has finished that, you will be able to do:

everything till step 5
run reginabox mirror npm-registry <full-path-to-custom-blob-store>/index.js true for fetch modules from the network instead of trying to pull everything down from npm first

@bengl, I wanted to make this setup more nice, without all of the installing steps, any suggestions? Would it be good to have all of this setting up be part of reginabox, so that devs only had to do npm i regianbox -g once and run the cli?

dignifiedquire commented 9 years ago

@diasdavid how about adding another module reginabox-cli which you can install globally and does all that work, without polluting the actual reginabox module?

dignifiedquire commented 9 years ago

So it would more like this:

$ npm i -g reginabox-cli
$ ipfs daemon 
$ reginabox mirror npm-registry
# internet connection lighting up because of all the traffic

daviddias commented 9 years ago

@dignifiedquire, regianbox is already a CLI :) so I guess we just need the changes on https://github.com/diasdavid/reginabox to go to https://github.com/yahoo/reginabox

dignifiedquire commented 9 years ago

hmmmmmmmm now that you say it like that cough ignore my comment, but it would still be nice to put all that manual work you mentioned in there or somewhere else to automate it more

dignifiedquire commented 9 years ago

It has begun

/Volumes/npm/custom-blob-store
❯ ls npm-registry |wc -l                                                                     
     185

ipfs / notes

npm on IPFS #2

Use Case 1: Raw Blob Store

Use Case 2: IPNS Mutable File System Thing

note: `mfs` is only available on IPFS dev0.4.0

Would something like this make sense? In a way, it would be useful to have a good way to access npm packages canonically, and have a trusted authority for feelgood-immutable npm@version names to ipfs objects.

Would something like this make sense? In a way, it would be useful to have a good way to access npm packages canonically, and have a trusted authority for feelgood-immutable npm@version names to ipfs objects.

ipfs / notes

npm on IPFS #2

Use Case 1: Raw Blob Store

Use Case 2: IPNS Mutable File System Thing

note: mfs is only available on IPFS dev0.4.0

Would something like this make sense? In a way, it would be useful to have a good way to access npm packages canonically, and have a trusted authority for feelgood-immutable npm@version names to ipfs objects.

Would something like this make sense? In a way, it would be useful to have a good way to access npm packages canonically, and have a trusted authority for feelgood-immutable npm@version names to ipfs objects.

note: `mfs` is only available on IPFS dev0.4.0