when running _coffee <filename.coffee> a fatal error occurs

[mreinstein]

dbits@bytesamurai:/mnt/wwwroot/Exceptions/server$ _coffee exceptions_srv.coffee Failed to find CoffeeScript from /mnt/wwwroot/Exceptions/server Error: Failed to find CoffeeScript from /mnt/wwwroot/Exceptions/server at Object.load (/usr/lib/node_modules/streamline/lib/util/coffee-loader.js:13:13) at Object.run (/usr/lib/node_modules/streamline/lib/compiler/underscored.js:65:54) at _run (/usr/lib/node_modules/streamline/lib/compiler/command.js:77:29) at Object.run (/usr/lib/node_modules/streamline/lib/compiler/command.js:107:3) at Object. (/usr/lib/node_modules/streamline/bin/_coffee:7:14) at Module._compile (module.js:446:26) at Object..js (module.js:464:10) at Module.load (module.js:353:31) at Function._load (module.js:311:12) at Array.0 (module.js:484:10)

I've scoured the internet looking for answers as to why this won't work, but haven't found anything. This used to work, but upon running npm update -g and getting updated code, this error seems to be occuring now. Any ideas why?

[bjouhier]

@aseemk: this seems to be due to the CS loader changes. I'm not too familiar with the Module functions that you used so I'm handing this to you

@mreinstein: I'd like to have Aseem take a look at it first. In the mean time, you should be able to get around it by installing coffeescript locally (for example in a /mnt/wwwroot/Exceptions/node_modules directory).

[mreinstein]

@bjouhier: thanks! I'll try that. Not sure if it helps, but I'm running coffeescript 1.3.3 If you need/want any logs just let me know.

[bjouhier]

I'm running 1.2.1-pre and it works but I'm not doing much with CS. We have a few streamlined CS files in our project but the bulk of our stuff is JS. Aseem changed the loader in 0.3.7 so I'd like to have his input before looking at it.

[aseemk]

Hey guys,

Sorry to hear you've run into this!

@mreinstein: would you mind sharing your $PATH, your $NODE_PATHS, and the output of npm ls -g? Thanks!

Also, what's your version of Node and npm?

[mreinstein]

@aseemk:

root@bytesamurai:~# echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

root@bytesamurai:~# echo $NODE_PATH /usr/lib/node_modules

root@bytesamurai:~# npm ls -g /usr/lib ├── coffee-script@1.3.3 ├─┬ connect-redis@1.3.0 │ ├── debug@0.7.0 │ └─┬ redis@0.7.2 │ └── hiredis@0.1.14 ├── exceptional-node@0.1.1 ├─┬ express@2.5.9 │ ├─┬ connect@1.8.7 │ │ ├── formidable@1.0.11 │ │ ├── mime@1.2.5 │ │ └── qs@0.5.0 │ ├── mime@1.2.4 │ ├── mkdirp@0.3.0 │ └── qs@0.4.2 ├── iniparser@1.0.5 ├── moment@1.6.2 ├─┬ mongodb@1.0.2 │ └── bson@0.0.6 ├─┬ redis@0.7.2 │ └── hiredis@0.1.14 └── streamline@0.3.6

node is v0.6.18 npm is 1.1.21

[aseemk]

Weird, your npm ls -g says you're running Streamline 0.3.6, but this change (and your error message) came in 0.3.7. Is something funky there?

Regardless, I think I realize what the issue is. When you run the _coffee binary, Streamline needs to load CoffeeScript, but CoffeeScript might not be present in Streamline's require() paths (see issue #84), so I made the change for Streamline to load CoffeeScript from the working directory, where it should be available if the code requires CoffeeScript.

I didn't foresee this global use-case: CoffeeScript isn't available from the code's location, but the developer runs it directly.

Lemme think about this and experiment with changing the behavior for just the "run _coffee directly" case. (It turns out my issue #84 was for the "compile" case, which is a different code path.)

Any thoughts on consequences if directly running a file with _coffee loads CoffeeScript relative to wherever _coffee is (e.g. the global dir) instead of local to the current working dir?

[aseemk]

Reference links btw:

• https://github.com/Sage/streamlinejs/commit/8b05d42efe29c78d1379d1a2fc72556fae97ff13#diff-2
• https://github.com/Sage/streamlinejs/blob/0.3.7/lib/compiler/underscored.js#L62-L65
• https://github.com/Sage/streamlinejs/blob/0.3.7/lib/util/coffee-loader.js

[mreinstein]

@aseemk: I went back to version 0.3.6 because that seems to work. It's only 0.3.7 that exhibits the problem. That's probably why the error is in 0.3.7 but my output shows 0.3.6, sorry about that.

I didn't foresee this global use-case

From what I've heard, running stuff directly in coffeescript, or streamlined coffeescript may be bad practice; I've heard a lot of people smarter than me suggest that everything should be compiled down to .js and run that way. Do you guys have any opinion on that? Is invoking _coffee directly to run my server code an unusual/bad use case?

[bigeasy]

I advocated for making _coffee an ordinary interpeter so that I would be able to write programs using Streamlined CoffeeScript and a shebang line. This is an easy way to write lots of little programs. I've been using Streamlined CoffeeScript to write utilities against web APIs or mixing it with bash for server administration utilities.

[bjouhier]

I've unpublished 0.3.7 to NPM. I'll publish the fix as a 0.3.8 when it is ready.

@mreinstein: there is no golden rule about compiling to .js or not. It all depends on your use case. If you are publishing a library that you have written with CS, it is better to compile it and publish the .js files. But if the CS code is for your own consumption, it might be better for you to just have the .coffee source files on disk. The _coffee command is designed to work this way. The only problem is that it got broken.

@aseemk: thanks for handling this. I did not have time to look at it yet but I have the impression that it is a path problem. Alan had set things up so that the path where _node was found was added to the module search paths, and I have the impression that this is what got broken.

[bjouhier]

@bigeasy Yes, it must support this use case, and this is why it need to have the directory where _coffee is installed in its module search path.

[bigeasy]

@bjouhier Actually, I don't believe I did anything to alter the search paths. It would just use NODE_PATH.

[bjouhier]

I was thinking of line 70 in your initial commit of underscored.js:

    mainModule.paths = require("module")._nodeModulePaths(path.dirname(filename));

I thought that this was the magic that made _coffee work when CS was globally installed. But maybe I'm wrong (I only have a superficial understanding of the module path issues).

[bigeasy]

I took it from coffee-script.coffee line 71. It sets the additional search path for the specific module, which is one step in a more thorough search when require is invoked.

In module.js, the search path for a loaded module is set on line 352. You can see on line 231 the function where a path like "coffee-script" would be resolved. It appends the module.paths, the additional paths for a specific module, to the path specified by modulePaths. That is the full search path.

Thus, that invocation of _nodeModulePaths is to override the initialization that would have been performed in Module.prototype.load. To replace the Module.paths for the _coffee executable with a Module.paths for the target Streamlined CoffeeScript program.

[bigeasy]

My goal was to make _coffee behave as coffee does. I assumed that CoffeeScript had enough time to come up with a good strategy for blending in.

Also, here's there Node.js module lookup procedure; Modules All Together. You're probably reading it this very moment already.

[aseemk]

Yes, you guys are definitely right that executing via the global _coffee needs to work — and the shebang use case is a great reminder.

At this point, I'm thinking that the best thing to do would be to try loading CoffeeScript locally first (i.e. relative to the file being run or compiled), and if not found, then try globally (i.e. relative to the _coffee binary). WDYT?

[bigeasy]

Module loading is controlled through NODE_PATH. Why does _coffee need its own module loading strategy apart from the one employed by Node.js and CoffeeScript?

[aseemk]

See issue #84.

[aseemk]

CoffeeScript is a simpler case: it doesn't need to load any other dependency.

[bigeasy]

I can't reproduce your example. I created a package.json as per #84.

{ "name": "project"
, "dependencies": { "neo4j": "~0.2.10", "streamline": "~0.3.4" }
, "version": "0.0.1" }

When I run npm install.

[alan@celje:~/junk/bar$] npm install
npm http GET https://registry.npmjs.org/neo4j
npm http GET https://registry.npmjs.org/streamline
npm http 200 https://registry.npmjs.org/neo4j
npm http GET https://registry.npmjs.org/neo4j/-/neo4j-0.2.11.tgz
npm http 200 https://registry.npmjs.org/streamline
npm http GET https://registry.npmjs.org/streamline/-/streamline-0.3.6.tgz
npm http 200 https://registry.npmjs.org/neo4j/-/neo4j-0.2.11.tgz
npm http 200 https://registry.npmjs.org/streamline/-/streamline-0.3.6.tgz

> neo4j@0.2.11 preinstall /home/alan/junk/bar/node_modules/neo4j
> npm install coffee-script@~1.3.1 && npm install streamline@~0.3.4

npm http GET https://registry.npmjs.org/coffee-script
npm http 200 https://registry.npmjs.org/coffee-script
npm http GET https://registry.npmjs.org/coffee-script/-/coffee-script-1.3.3.tgz
npm http 200 https://registry.npmjs.org/coffee-script/-/coffee-script-1.3.3.tgz
coffee-script@1.3.3 ./node_modules/coffee-script
npm http GET https://registry.npmjs.org/streamline
npm http 304 https://registry.npmjs.org/streamline
streamline@0.3.6 ./node_modules/streamline
npm http GET https://registry.npmjs.org/http-status
npm http GET https://registry.npmjs.org/request
npm http 200 https://registry.npmjs.org/http-status
npm http GET https://registry.npmjs.org/http-status/-/http-status-0.1.1.tgz
npm http 200 https://registry.npmjs.org/http-status/-/http-status-0.1.1.tgz
npm http 200 https://registry.npmjs.org/request
npm http GET https://registry.npmjs.org/request/-/request-2.1.1.tgz
npm http 200 https://registry.npmjs.org/request/-/request-2.1.1.tgz

> neo4j@0.2.11 postinstall /home/alan/junk/bar/node_modules/neo4j
> npm run build

> neo4j@0.2.11 build /home/alan/junk/bar/node_modules/neo4j
> _coffee -c lib/

streamline@0.3.6 ./node_modules/streamline

neo4j@0.2.11 ./node_modules/neo4j
├── http-status@0.1.1
└── request@2.1.1

[aseemk]

Because I worked around it in the Neo4j library by explicitly installing Streamline locally preinstall. Try an earlier version; check the changelog.

On Tuesday, June 12, 2012, Alan Gutierrez wrote:

I can't reproduce your example. I created a package.json as per #84.

{ "name": "project"
, "dependencies": { "neo4j": "~0.2.10", "streamline": "~0.3.4" }
, "version": "0.0.1" }

When I run npm install.

[alan@celje:~/junk/bar$] npm install
npm http GET https://registry.npmjs.org/neo4j
npm http GET https://registry.npmjs.org/streamline
npm http 200 https://registry.npmjs.org/neo4j
npm http GET https://registry.npmjs.org/neo4j/-/neo4j-0.2.11.tgz
npm http 200 https://registry.npmjs.org/streamline
npm http GET https://registry.npmjs.org/streamline/-/streamline-0.3.6.tgz
npm http 200 https://registry.npmjs.org/neo4j/-/neo4j-0.2.11.tgz
npm http 200 https://registry.npmjs.org/streamline/-/streamline-0.3.6.tgz

> neo4j@0.2.11 preinstall /home/alan/junk/bar/node_modules/neo4j
> npm install coffee-script@~1.3.1 && npm install streamline@~0.3.4

npm http GET https://registry.npmjs.org/coffee-script
npm http 200 https://registry.npmjs.org/coffee-script
npm http GET
https://registry.npmjs.org/coffee-script/-/coffee-script-1.3.3.tgz
npm http 200
https://registry.npmjs.org/coffee-script/-/coffee-script-1.3.3.tgz
coffee-script@1.3.3 ./node_modules/coffee-script
npm http GET https://registry.npmjs.org/streamline
npm http 304 https://registry.npmjs.org/streamline
streamline@0.3.6 ./node_modules/streamline
npm http GET https://registry.npmjs.org/http-status
npm http GET https://registry.npmjs.org/request
npm http 200 https://registry.npmjs.org/http-status
npm http GET
https://registry.npmjs.org/http-status/-/http-status-0.1.1.tgz
npm http 200
https://registry.npmjs.org/http-status/-/http-status-0.1.1.tgz
npm http 200 https://registry.npmjs.org/request
npm http GET https://registry.npmjs.org/request/-/request-2.1.1.tgz
npm http 200 https://registry.npmjs.org/request/-/request-2.1.1.tgz

> neo4j@0.2.11 postinstall /home/alan/junk/bar/node_modules/neo4j
> npm run build

> neo4j@0.2.11 build /home/alan/junk/bar/node_modules/neo4j
> _coffee -c lib/

streamline@0.3.6 ./node_modules/streamline

neo4j@0.2.11 ./node_modules/neo4j
├── http-status@0.1.1
└── request@2.1.1

---

Reply to this email directly or view it on GitHub: https://github.com/Sage/streamlinejs/issues/105#issuecomment-6271658

[bigeasy]

Okay. You solved the problem without a special-to-Streamline.js-only module loading algorithm. You used the existing module loading algorithm and the features of NPM.

Why does Streamline.js need to be different?

[aseemk]

Haha. I'm about to enter the subway. Will respond later.

On Tuesday, June 12, 2012, Alan Gutierrez wrote:

Okay. You solved the problem without a special-to-Streamline.js-only module loading algorithm. You used the existing module loading algorithm and the features of NPM.

Why does Streamline.js need to be different?

---

Reply to this email directly or view it on GitHub: https://github.com/Sage/streamlinejs/issues/105#issuecomment-6271846

[aseemk]

Here are some quick points, Alan:

• I didn't solve the problem, I worked around it.
• The workaround involves manually installing a dependency — defeating part of npm's purpose — and duplicating the semver of that dependency. Hardly elegant.
• Failing to implement that workaround exposes a module to a leaky abstraction, where it can work fine during development but can fail randomly in other environments depending on the surrounding setup. And indeed, someone else ran into this same issue with Streamline and CoffeeScript two weeks after I first came across it, FWIW.
• The issue I was trying to address is simply: if code asks to compile Streamlined CoffeeScript, it seems to me that Streamline shouldn't be using its CoffeeScript dependency (which isn't even specified, because CoffeeScript isn't actually a true dependency of Streamline), but rather the code's CoffeeScript dependency.
• My fix isn't achieved via some fancy algorithm. It's really the same idea as your above comment — a file's dependencies should get resolved to the file, not to the _coffee binary. In fact, it uses the same native _nodeModulePaths() method, which is indeed effectively NODE_PATH.

Hope this clarifies things.

[aseemk]

To clarify one last thing: it seems to me that the best way to fix this would be to first try the local dependency, and if that doesn't work, then simply require() it regularly (i.e. relative to Streamline).

Shebang scripts will work fine then, if CoffeeScript is installed globally.

Edited to remove a phrase that could be construed as snark. =)

[bjouhier]

Hi Assem and Alan,

I'm not sure I can help here. I can follow the discussion but I don't have much expertise on the NPM and node path issues.

I didn't specify the CoffeeScript dependency in package.json because I don't want to force the JS guys to install CS if they won't be using it. Maybe it would be better to have a separate streamline-coffee package that just wraps streamline and CS together. But streamline is a language tool, not a library so I'm expecting that the CS guys will have CS installed anyway.

Aseem, I suggest that you fix the problem with your proposed strategy. Then it would be great if Alan could install it and test it before I publish the fixed version to NPM.

[aseemk]

Agreed — thanks Bruno!

Edit (man, I need to stop immediately submitting after typing, haha): I didn't mean to imply CoffeeScript should be a dependency; you're absolutely right that it shouldn't be and is not. (It's more like an optional dependency, but not what npm considers optional dependencies.)

Having a separate package for CoffeeScript support wouldn't be a bad idea, but I think CoffeeScript is mainstream enough (as evidenced by direct support for it in other popular tools and libraries) that supporting it directly is nice.

[mreinstein]

@bjouhier btw, I wanted to say thank you for creating streamlinejs. When I was first evaluating nodejs as a tool, I was put off by the "pyramid of doom" pattern that shows up in callback rich libraries. I looked at many other solutions, including some promise based stuff, and streamline seems to me like the most elegant solution. Wondering, do you have a blog?

[bjouhier]

Aseem,

No problem. I did not take your comment as meaning that CS should be a dependency but somehow it gave me an opportunity to explain why it is not. And actually, I had a similar problem with fibers. Streamline does not install it. So, I'm now wondering if the wrapper package is such a good idea because then I'll need another wrapper for fibers and another one for CS + fibers. So what I would really need here is a way to specify optional dependencies (or install options) as NPM command line options. I don't think NPM supports this but I could ask for it.

Mike,

Thanks. I have a blog where you will find more info about streamline. The first article to read is the Harry story. The YR Javascript article is much harder to swallow but it explains how the streamline transform works.

[aseemk]

Bruno: ah, you're right, Fibers is subject to the same issue! It'd be easy for me to generalize the CoffeeScript loader to load other dependencies. Want me to update the require('fibers') to that too?

Re: npm: ultimately, it's not quite just about Streamline specifying the dependency — we should be using the version of Coffee/Fibers that the user wrote his/her code for. (There's more to it than that technically — Streamline is also using their APIs, so ideally both clients could coordinate the dependency.)

Short of npm fixing this internally or building a whole system to coordinate multiple clients sharing the same dependency (won't happen), seems like a good fix to just try to load the dependency from the user's code first. =)

[bjouhier]

Well, I was only concerned about the installation issue re. fibers. And I get your point about CS. It is certainly better to keep streamline very decoupled from CS so that people can use whatever CS version they like.

I don't think that we need to change anything in the way fibers is required. Fibers is much easier to handle than CS because it does not install any require hooks. So, it does not need to be handled in the loader, it can be required from the runtime module, like any other library.

[aseemk]

Hmm, tested this out real quick and I can produce the same issue — if Streamline is in a parent directory, but Fibers is local, running the compiled code fails:

$ mkdir -p /path/to/parent
$ cd /path/to/parent

# in parent:
$ mkdir node_modules
$ npm install streamline
$ mkdir node_modules/child
$ cd node_modules/child

# in child:
$ mkdir node_modules
$ npm install fibers
$ cat > foo._js <<EOF
  console.log('hello...');
  setTimeout(_, 1000);
  console.log('...world');
  EOF
$ _node --fibers -c foo._js
$ node foo.js

node.js:201
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
Error: Cannot find module 'fibers'
    at Function._resolveFilename (module.js:332:11)
    at Function._load (module.js:279:25)
    at Module.require (module.js:354:17)
    at require (module.js:370:17)
    at Object.<anonymous> (/path/to/parent/node_modules/streamline/lib/fibers/runtime.js:7:1)
    at Module._compile (module.js:441:26)
    at Object..js (module.js:459:10)
    at Module.load (module.js:348:31)
    at Function._load (module.js:308:12)
    at Module.require (module.js:354:17)

Like with CoffeeScript, this can happen if module A specifies Streamline as a dependency, but not Fibers, and requires module B, which specifies Streamline and Fibers; in that case, npm won't install Streamline local to module B, so Fibers won't resolve relative to Streamline.

Edit: npm ls output:

/path/to/parent
├── streamline@0.3.7a 
└─┬ child@0.0.0 
  └─┬ fibers@0.6.7 
    └─┬ ...

[bjouhier]

You're right. Something's leaky here. So, yes, fix it if you can use the same pattern

[bigeasy]

Failing to implement that workaround exposes a module to a leaky abstraction, where it can work fine during development but can fail randomly in other environments depending on the surrounding setup. And indeed, someone else ran into this same issue with Streamline and CoffeeScript two weeks after I first came across it, FWIW.

The issue I was trying to address is simply: if code asks to compile Streamlined CoffeeScript, it seems to me that Streamline shouldn't be using its CoffeeScript dependency (which isn't even specified, because CoffeeScript isn't actually a true dependency of Streamline), but rather the code's CoffeeScript dependency.

My fix isn't achieved via some fancy algorithm. It's really the same idea as your above comment — a file's dependencies should get resolved to the file, not to the _coffee binary. In fact, it uses the same native _nodeModulePaths() method, which is indeed effectively NODE_PATH.

First of all, you keep saying "leaky abstraction" as if it is a given that people are going to understand what you're talking about. "Failing to implement that workaround exposes a module to a leaky abstraction." In essence, this pull request is solving a leaky abstraction. The point of Joel's leaky abstraction post is that leaks in abstractions are inevitable.

The law of leaky abstractions means that whenever somebody comes up with a wizzy new code-generation tool that is supposed to make us all ever-so-efficient, you hear a lot of people saying "learn how to do it manually first, then use the wizzy tool to save time." Code generation tools which pretend to abstract out something, like all abstractions, leak, and the only way to deal with the leaks competently is to learn about how the abstractions work and what they are abstracting. So the abstractions save us time working, but they don't save us time learning.

Joel's parable is about how you can't _not_ know what's behind the abstraction layer. He's not defining a class of problems to be solved. He's not saying, if you see an abstraction leaking, you've got to plug that leak.

Joel is saying, if you see an abstraction leaking, chill out, it happens.

But, that's not happening here — neither the leaking, nor the chilling.

Streamline.js is now taking on complexity to make NPM do something that Issac does not want it to do, act as a general purpose, automated build environment. When you read his response to your inquiries about global versus local require he says...

This whole discussion is a little bit absurd.

If I had a time machine, and could go back to 2010 and do it again, I would do all that I could to prevent require.extensions from even being added. At the time, I thought it was a good idea. In retrospect, I was wrong. It has only caused problems.

Here's a better approach than require('coffee-script') to set up the require hook: publish and run JavaScript.

Publish JavaScript means compile to JavaScript on your development machine, that you know to be configured correctly for Streameline.js compilation, and publish that JavaScript to NPM. NPM is not a continuous integration system. It is not meant to be. Issac shows no signs of transforming it into one.

There is no "leaky abstraction". NPM does not abstract a continuous integration system. It is not a "leaky abstraction" that forces you to jump through hoops to make it act as one.

Currently, _node and _coffee run find as interpreters in a POSIX environment. If I were to make any changes to the invocation of _node and _coffee, it would be for them to have command line option parity with CoffeeScript, a project with a lot of wisdom and experience.

[aseemk]

Hey Alan, I don't really have the energy to engage in a philosophical debate right now. I don't mean to be dismissive.

What's your issue here? _node and _coffee will continue to run perfectly fine as interpreters after #107.

[bigeasy]

This behavior does not belong in Streamline.js.

[aseemk]

What problem does it cause exactly? I've shown you two very real problems it solves.

[bigeasy]

No, you haven't. You're trying to use NPM as a continuous integration system. That is not what it was mean to do.

The problem that it causes is code bloat, complexity, and an edge case that will need to be supported as a legacy.

[aseemk]

I guess the crux of this philosophical debate boils down to whether you think using a require() hook and publishing to npm source rather than compiled JS is a good thing.

I'm sure I can't convince you either way; I can simply share that we've found both to be valuable, and both are supported by node and npm today, so it's going to happen. (And it indeed happens; e.g. CoffeeScript is the second-most depended-upon package in npm.)

Given that it happens, it seems rational to me to support it in Streamline. I guess I don't see this as having any significant amount of code bloat or complexity. (In fact, it seems elegant to me: when loading any other compiler dependency, try loading it local to the code first. That's all this is.)

On the flip side, having this will prevent unexpected errors and prevent people from wasting a bunch of time figuring out why something isn't building. That's what I ran into, and that's what the other person I mentioned ran into.

If you're curious why we like using require() hooks or publishing source to npm: we love not having to check compiled JS in (reduces diff noise), and we like that our compiler dependencies are just like our other dependencies in that they're "live" — as the compilers get updated, our modules get those updates (within their semver specs).

[bigeasy]

You don't have to check in your compiled source in order to publish it to NPM.

The NPM hooks are how this is done for CoffeeScript. http://stackoverflow.com/a/8671733/90123 Your hook based build that works with 0.3.6 is the correct way to do this.

Streamline.js should not have it's own set of require logic that is different from the require logic of node. It is one more thing to make Streamline.js foreign to noders.

[aseemk]

Hey Alan, we're going in circles here.

I'm confused by the Stack Overflow post you link to. You can see that someone else is using a postinstall script to compile CoffeeScript, and that it should be as simple as cake build or similar. That's exactly what I had done with _coffee, and it was subject to failure depending on the parent's dependencies.

With this fix, _coffee in a postinstall script "just works" as you'd expect it to.

This isn't fancy require logic — it's really as simple as "require relative to the code" just like both Streamline and CoffeeScript already do for regular dependencies.

Ultimately, it doesn't seem like I can convince you here, which is fine; we'll just agree to disagree on this.

Btw, I didn't mean to imply compiled JS needs to be checked in in order to publish it to npm. I was explaining why we like require() hooks (which you brought up). I explained we don't publish compiled JS to npm in order to have the compilers themselves be treated just like other regular dependencies.

[aseemk]

Unrelated to this actually, I do want to thank you for contributing underscored.js in the first place.

Using the ._js and ._coffee extensions (which I was wary of at first) and the _node and _coffee binaries really has been awesome — much nicer than the previous system. =)

Thanks!

[bjouhier]

What matters most to me is:

1) Are we introducing a regression here? 0.3.7 was a clear regression but I have the impression that we are back on tracks and that everything that worked in 0.3.6 should work again.

2) Are we changing the way require works in streamlined modules? I have the impression that the new code is tweaking require so that the compiler can load coffee-script and so that the runtime can load fibers but that streamlined modules will continue to use the regular require, not the tweaked one.

So unless I'm mistaken on these or I'm missing something big, I think that this fix is worth it. It makes things work "out of the box" in situations where it did not before.

[aseemk]

Indeed, my apologies for the 0.3.7 regression!

And indeed, with the singular exception being the load of fibers, the streamlined code itself continues to use the regular require().

The special-case of fibers is only needed because the require() call comes from within a file that's located within the Streamline library — the call isn't injected into the compiled code.

(Note that if a regular require() call were injected into the compiled code, the shebang use case would fail for the fibers case if fibers wasn't installed local to the file, assuming that global installs aren't on the require() path, as is the case by default.)

[aseemk]

On that note, is there even any way to specify fibers for shebang files? Does e.g. #!/usr/bin/env _coffee --fibers work / work consistently?

[bigeasy]

No. It does not work much at all. Env will get the what comes after env as a single string, so env will get a single argument "_coffee --fibers".

My take on this issue was that if you're scripting with the shebang line, you're not going to want to concern yourself with performance issues. You're going to want a reasonable default. If the reasonable default does not work, then you're going to need to take more control of your environment than the shebang line will permit. You're going to have to compile your Streamlined code.

[aseemk]

I thought so too, so I was trying to come up with other clever ways to achieve it (like #!/usr/bin/which _coffee--fibers), but it turns out you can pass arguments, and it works!

http://en.wikipedia.org/wiki/Shebang_(Unix)#Syntax

Cool. =)

[bigeasy]

You can pass a single argument to the shebang executable. If you're using env it won't work.

[aseemk]

It seems to work.

Crazy example as an input of compiling the file instead of running it:

#!/usr/bin/env _coffee --fibers -c
console.log 'hello world'

#!/usr/bin/env _coffee --fibers -c#!/usr/bin/env _coffee --fibers -c
/*** Generated by streamline 0.3.6 (fibers) - DO NOT EDIT ***/(function() {
  console.log('hello world');
}).call(this);

The shebang getting duplicated in the output is interesting, but hardly important. =)

[aseemk]

Ah, I see what the issue is: apparently this doesn't always work on Linux. I'm on Mac OS X, where it works fine.

http://en.wikipedia.org/wiki/Shebang_(Unix)#Portability