node support

[timoxley]

I'd just like to put it out there that it'd be nice if we could do away with npm(1) entirely (in the nicest way possible, it'd be good to have some healthy competition in this area).

[defunctzombie]

I would support that line of thinking if the new thing is practically and ideally measurably better. Also free of single entity control might be reason enough. On Apr 10, 2014 12:20 PM, "Tim Oxley" notifications@github.com wrote:

I'd just like to put it out there that it'd be nice if we could do away with npm(1) entirely.

Reply to this email directly or view it on GitHubhttps://github.com/component/duo/issues/7 .

[juliangruber]

+1

[jonathanong]

yeah i'm thinking only using ES6/WHATWG standards. do stuff like import "//component.io/component/emitter/^1.0.0/index.js" and crawl all the dependencies as single files, including CSS, HTML, images, etc. then have the CDN SPDY Push the entire dependency tree to the client. no need for any .json packages anywhere. i think this is how browsers will work anyways, so this stuff should work and without a package manager (unless you want to do stuff like bundling)

[jonathanong]

also, i'm not sure how to do node support for any new package manager without actually forking node. i wonder how they're going to implement es6 modules.

[defunctzombie]

Personally I would ignore es6 modules but I guess this is gonna get into some uncharted unpopular territory On Apr 10, 2014 2:29 PM, "Jonathan Ong" notifications@github.com wrote:

also, i'm not sure how to do node support for any new package manager without actually forking node. i wonder how they're going to implement es6 modules.

Reply to this email directly or view it on GitHubhttps://github.com/component/duo/issues/7#issuecomment-40121128 .

[jonathanong]

meh i don't think cjs -> es6 modules is a big deal (or vice versa for backwards compatibility). i'm just hoping people stop using RJS and creating UMD wrappers.

[Swatinem]

i'm just hoping people stop using RJS and creating UMD wrappers.

:+1:

[stagas]

wouldn't that be a trio then? :P node will have to change the way packages are stored anyway because of long path issues on windows, so maybe this is a chance for global consensus on package path resolution https://github.com/joyent/node/issues/6960

[matthewmueller]

Actually this was my original goal :-P.

Node support can be done with require rewriting and symlinking. But it does require a lot of upfront work in the installer to walk and rewrite requires.

[lancejpollard]

:+1:

[Gozala]

I would really love to use duo as an alternative to npm! I think it is also going to make duo a lot more appealing for library developers.

@MatthewMueller

I think you can get away without require rewriting and symlinking actually. I did it once with project called teleport. Since node lookup is basically based on looking into node_modules lookup all you need is to arrange installed files so that node could find them. If you guys will be willing to change following syntaxrequire('yields/shortcuts@0.0.1:/index.js) to require('yields/shortcuts@0.0.1/index.js) there is no reason why duo would not be able to just manage node_modules folder for downloading dependencies. As a matter of fact you can just install all the nested dependencies into same top node_modules folder to reduce all the annoying duplication that comes with npm.

[Gozala]

@MatthewMueller on your request (in IRC) I made an example repo illustrating how node support can be implemented: https://github.com/Gozala/duo-node/tree/master/app You can also run nodeduo-node/app/main.jsto see that it actually works. All the paths I'll mention below will be relative toduo-node/app/`

If you look at the imports in the main.js there are two:

var foo = require("gozala/foo")
var bar = require("gozala/bar@2.4")

On first require of gozala/foo node will perform following lookups in order to find a module:

./node_modules/gozala/foo.js
./node_modules/gozala/foo/package.json#main 
./node_modules/gozala/foo/index.js

Note: Second line means see if package.json is present, than load a module that is specified in under a main key.

In this case last lookup will succeed and load ./node_modules/gozala/foo/index.js. Which is great since dou can simply download modules form master branch of https://github.com/Gozala/foo into ./node_modules/gozala/foo.

Second import of gozala/bar@2.4 in main.js is no different from node perspective than gozala/foo although duo can install modules from tag 2.4 of https://github.com/Gozala/foo into ./node_modules/gozala/bar@2.4 :)

Things do get slightly tricky for the nested dependencies like in case of ./node_modules/gozala/bar@2.4/index.js which depends on gozala/bar@2.3 module. Here node will try following lookups to find a required module:

./node_modules/gozala/bar@2.4/node_modules/gozala/bar@2.3.js
./node_modules/gozala/bar@2.4/node_modules/gozala/bar@2.3/package.json#main
./node_modules/gozala/bar@2.4/node_modules/gozala/bar@2.3/index.js
./node_modules/gozala/node_modules/gozala/bar@2.3.js
./node_modules/gozala/node_modules/gozala/bar@2.3/package.json#main
./node_modules/gozala/node_modules/gozala/bar@2.3/index.js
./node_modules/gozala/bar@2.3.js
./node_modules/gozala/bar@2.3/package.json#main
./node_modules/gozala/bar@2.3/index.js

Notice the pattern ? It basically walks the tree up to system root looking for desired modules under node_modules dir at each level. There for last lookup will succeed. Which is great cause duo can do exactly what it did in the previous case.

In nutshell due can stick to it's flat directory structure for dependencies and node will carry on with it's recursive lookup until app level dependencies are reached. This has following advantages over the current standard:

1. Dependencies can be shared and will not be duplicated. Note that npm does opposite it duplicates dependencies in clam to make it more reliable. But unlike in npm here you specify exact version and if two libraries depend on same version there is no problem of sharing. And if it becomes a problem for your lib well just clone your dependency and require that. I think npm is afraid of shared dependencies & they end up copy them all over place while all they needed is to do is make require(x) always return the same regardless of when you do it. Git definitely helps here.
2. No configurations.
3. No publishing to github then npm.
4. No central authority.
5. End to a naming conflicts.

Note that in nutshell what due will need to do is treat node_modules folder of the package it's running from as registry where all modules are installed that's it.

In terms of tradeoffs due would have to adjust syntax slightly (as node will choke on :) which is from require('yields/shortcuts@0.0.1:/index') to `require('yields/shortcuts@0.0.1/index') which I even find more intuitive to be honest.

[Gozala]

As I already pointed out earlier sometime in the past I did a prototype of a tool that did something very similar. Although it tried to stay open and independent of github or other service and just supported URLs:

require('foo.org/bar') //  http://foo.org/bar.js > ./node_modules/foo.org/bar.js
require('!bla.org/baz') // https://bla.org/baz.js > ./node_modules/bla.org/baz.js

I'd be happy to help out with adding a node support as I really think something better than npm should come along & duo is on a right track.

[Gozala]

Oh and if we do end up implementing this we can then make duo use doe for it's dependency management and avoid npm install duo :)

[Mithgol]

For such “Node support” it might also be useful to leave Node's require() method intact and rename Duo's require() to duo() with an additional bonus of being shorter (three letters instead of seven).

[timoxley]

rename Duo's require() to duo()

why?

[Mithgol]

Otherwise you won't be able to call Node's require() because require() would mean Duo's method, as far as I understand. (Please correct me if I am wrong.)

[dominicbarnes]

The vastly better solution to that problem is to include a "no conflict" mode. (like jQuery does)

I'm thinking a global DUO var will be unavoidable (albeit very helpful) as time goes on.

[Mithgol]

Nope, that “no conflict” mode should be the only available mode.

(Unless someone might actually have a reason to replace Node's require() by Duo's despite the trouble that would be caused inevitably. Can you think of one such reason?)

[stephenmathieson]

@Mithgol that's the idea.

[Gozala]

I'm not sure if this discussion went completely sideways, but all I was describing works in node as is. All node would have to do is place installed files as described to be compatible with node. So I don't really support or like idea of changing require or coming up with some alternative.

[matthewmueller]

@Gozala +1

If we can't support node with what we already have, it's probably best suited for another project. Keep in mind this is a front-end package manager which means it will hopefully also be used outside of the node.js world.

I don't want to couple Duo to the decisions that Node.js made. I think it's possible to get Node working and while changing the recursive directory structure to a flat directory structure. In fact, I think @Gozala was able to get this working in https://github.com/Gozala/duo-node.

[Mithgol]

Ah, I understand the concept of @Gozala's demo now. It's not necessary to rename Duo's require() method because you won't need it at all (you would use only Node's require() method) anyway.