Make traceur (maybe all traceur-compiled modules) loadable via node 'require'

[GoogleCodeExporter]

I believe it might not be much harder to do this for all modules, so
I've tentatively extended the scope of the bug.

  For those just joining: Currently traceur uses an 'eval' hack to
  load into node. This loses line numbers in stack traces, and some
  flexibility.

  Proof-of-concept (way out-of-date currently):

    Make bin/runtime.js and bin/traceur.js
    run natively in both node and browser.
    https://codereview.appspot.com/6745048/

The goal in a nutshell is (as I see it)

  $ cat H.js
  module H {
    export function h(n) {
      return 'hello ' + n;
    }
  }
  $ ./traceur --out H.out.js H.js
  $ node
  > var runtime = require('bin/traceur.js');
  > traceur.runtime.setupGlobals(global); // See below.
  > var H = require('./H.out.js').H;
  > H.h('world');
  'hello world'
  >
  $ cat H.out.html
  <script src="runtime.js"></script>
  <script src="H.out.js"></script>
  <script>console.log(H.h('world'));</script>
  $ my-browser H.out.html

Caveats:

Because modules can't affect the loading code's environment, you
need an extra step to install the polyfills for running compiled
traceur code in the current context:

  var traceur = require('bin/traceur.js');
  traceur.runtime.setupGlobals(global);

There may be other unforseen problems, obviously, so maybe have this
as an optional output format, not default.

Original issue reported on code.google.com by usrbi...@yahoo.com on 26 Feb 2013 at 10:10

[GoogleCodeExporter]

One thing to add is that modules are about to change. In a nutshell:

module "a/b/c" { ... }

is a named module that is always available under that name, no matter where it 
is required from.

module "./a/b/c" { ... }

is a relative module and it is resolved like a URL (also "../a/b/c", "/a/b/c", 
"http://a/b/c")

So when you do `import {x} from "a/b/c"` it will always get the named module 
but `import {x} from "./a/b/c"` will resolve the name based on the current 
module. (nested modules are not allowed any more)

----------------------------

With that in mind, I think a more reasonable approach is to do something like:

module 'abc' {
  export var x = 42;
}

// compile

<script src="compiled.js"></script>
var moduleInstance = traceur.loader.get('abc');
var x = moduleInstance.x;
</script>

-------------------------------

Node.js allows us to hook a transpiler step into the loader. I played around 
with this before but it broke down on our compilation for external modules 
being async. Node.js needs sync require.

Original comment by arv@chromium.org on 27 Feb 2013 at 5:27

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Just to make things clear, these are no longer legal?

  // Illegal. Needs to be:
  //   module 'M1' { ... }
  module M1 { ... }
  // Illegal. M2M is a nested module.
  module M2 {
    export module M2M { ... }
  }
  // Illegal. M3M is still a nested module.
  module M3 {
    import M3M from 'M3M.js';
    export M3M;
  }

And for loading:

  // file.js
  module file { ... }

  // file.html
  <script src="file.out.js"></script>
  <script>
    // Undefined reference. 'file' no longer magically appears upon
    // load via script tag.
    file.func();
  </script>

----

With the current implementation, name conflicts are handled
silently, with the last declaration winning (normal JS behavior).

With the new style (reminds me of node or python), does this become
a runtime error?

For node or python, there is a search path for named modules,
eventually searching the importing file's local directory. Is this
similar to how harmony modules is going to be? Or are the directory
separators essentially meaningless semantically?

  // So this never means 'c.js' in subdir 'b' in subdir 'a'?
  module "a/b/c" { ... }

This seems to run counter to both node and python, which is not to
say it's bad, but that it's counter to common and familiar usage. If
there really is a compelling reason, then it's good, but otherwise,
familiar seems better.

----

Didn't know about the node transpiler hooks. Also have not really
tried traceur's async module loading, though kind of vaguely aware
of it from working on those files.

I'm not sure how invisible we can realistically expect the traceur
layer to be. Currently, I was thinking of explicitly doing 'require'
or direct module access depending on the environment.

But it seems a lot more complicated to do
  import 'M' as M;
and have it do the right thing across all environments. And node is
actually 3 environments: direct node, node via 'require', and node
via repl. (Okay, some of the differences might not matter. I haven't
thought this completely through.)

Original comment by usrbi...@yahoo.com on 27 Feb 2013 at 7:18

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

I got some details wrong "a/b/c" is resolved relative to the current module 
loader, not relative to the current file.

Original comment by arv@chromium.org on 28 Feb 2013 at 5:14

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

The "relative to the loader" makes more sense. Started
wading through harmony:modules.

  [some reading...]
  http://wiki.ecmascript.org/doku.php?id=harmony:module_loaders

  if there is not currently an entry in the module instance
  table for its canonical URL, the module must be fetched.

So it looks like

  module 'a/b/c' { ... }

probably creates an "entry" in the "module instance table".
But if it doesn't exist, then the Loader prepends its own
baseURL and fetches it.

Being able to specify module pathnames directly is
definitely different from node and python, but kind of makes
sense in the world of CDNs with potentially arbitrary URLs.

----

Since '/be' takes a lot of inspiration from python, it's not
surprising we see a global 'System' object taking the role
of a central registry for modules.

If JS did nothing more than copy python's module system, I
don't see how you could go too wrong, at least in node-style
environments.

On the web, your CDN may go down, and then you need
fail-over, and maybe checksum or signature verification of
your loaded modules just to make sure that no one has
hijacked the backup CDN.

  But then again, I guess HTTPS takes care of that --
  theoretically. But more of a pain to set up, so not many
  do it. And for CDNs, doesn't HTTPS preclude caching?

Maybe I'm being paranoid there. But even if not everybody
verifies to that level, the option to do so should (maybe
is; haven't read the wiki enough to fully understand)
probably be available.

Original comment by usrbi...@yahoo.com on 28 Feb 2013 at 10:14

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Would it be possible and/or spec-conforming to load the modules imported 
ahead-of-time, using require (instead of loading and inlineing the file), 
output destructuring code in case import * is used (or make a frozen object 
with the properties found), and generate a require call (the second call will 
used the module cached from the ahead-of-time call)?

That would allow things like "module fs from 'fs';" or "import * from 'http';" 
or "import Command from 'commander';" to work nicely.

It's possible to wrap around the .js module extension handler (I doubt you 
would want a different extension), to compile with traceur.

Original comment by edy.b...@gmail.com on 27 Mar 2013 at 1:23

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

There are two possibilities:

- Compile one file that will work unmodified in both browser and node.

  This obviously wouldn't work for native node modules, but should work
  with traceur-compiled modules.

- Have a "node" output target that outputs node modules and turns
  imports into 'require'.

I'd prefer the first possibility if it's possible, because it makes
traceur simpler to use. The output files would Just Work (TM) anywhere.

For source-map, we just wrapped it and used it like a normal traceur
module. That works fine for a few libraries, but isn't scalable.

----

The modules spec requires a lot of things, including a 'System' object
to play the role of 'sys' in Python, and a Loader interface. So a
wrapper around node that uses that interface would probably be
conforming.

  This is the best general solution for node. Have a node-specific
  Loader. Note that we haven't implemented System yet, and I'm not sure
  how spec-conforming our Loader objects are.

The spec is in flux, and I'm not good at being a spec lawyer, so don't
take this as definitive.

Original comment by usrbi...@yahoo.com on 27 Mar 2013 at 5:20

• Added labels: ****
• Removed labels: ****