Transform `examples/js` to support modules

[ghost]

The main source code has recently been ported to ES2015 modules (see #9310); however, some parts are still relying on global namespace pollution and have thus become unusable.

Specifically the things in /examples/js/ haven't been transformed to support modules yet; this makes them currently unusable from within environments such as webpack or SystemJS.

Edit, 11-20-2016: They can be used, but it requires a lot of setup: SystemJS, webpack. If you need assistance with these: Sorry, but this is the wrong thread to announce it! 😄

[backspaces]

I've had good luck using System.js with commonjs and global modules. The main issue here is the direct install of the controls into THREE. My guess is that if the controls first built their module object in the global space, then installed it in THREE in the current namespace, it might work, and not break prior code.

I plan to grab the controls, and just run a simple script over them to install themselves in the global space first, then install them in THREE. Some fussing around is bound to work. And no, a script tag isn't OK. The locality of Modules is great, the import is where its needed. I'll bet a large percent of html files with over 20 script tags have scripts that are no longer used!

[killroy42]

I use this module-loader.js With this module loader it can work like this: index.html:

<script src="/vendors/THREE/three_r83dev.js"></script>
<script src="https://gist.githubusercontent.com/killroy42/b8cbbd90b1209e8d8dec817ef88e8730/raw/dea5a7db1a26349b73e14f7f3ff5bca7c5408662/module-loader.js"></script>
<script src="/vendors/THREE/controls/OrbitControls.js"></script>
<script src="script.js"></script>

script.js:

const {
  OrbitControls,
  Box3, BoxGeometry, MeshBasicMaterial, SphereGeometry
} = require('THREE');

That's what I use...

[backspaces]

@mrdoob: This is a serious problem for those of us who have gone to es6 and modules. More serious than a simple "issue". It's the tipping point for Three: how to be "forward compatible"!

Modules are the most important problem because they completely change the way projects encapsulate their code. You have brilliantly gone to modules internally, Yay!, but have difficulties with the controls due to their directly installing into Three. This is fine, IMHO, it is Javascript after all!

There really are two problems:

1 - Three itself. I would prefer direct access to Three modules so that the module loader gives me only what I need. This, with an http/2 CDN, would likely improve Three performance.

2 - Legacy. AFAIK only controls are the problem? If so, I believe the problem can be solved by simply removing the direct installation into Three and letting them be what Guy Bedford (system.js) calls "global" modules. .. i.e. modules that simply install themselves in window. Those system.js can handle, witness the ability to import Three.

If a simple modification of controls can both remain backward compatible but load as an import, that's cool.

But face it: like all projects, Three has to cope with both backwards and forwards compatibility. Suggestion: Talk to Guy to see what other's have had success with, OK? He's very approachable.

[mrdoob]

Suggestion: Talk to Guy to see what other's have had success with, OK? He's very approachable.

Maybe you can do that for us?

[mrdoob]

Today I worked on https://github.com/mrdoob/model-tag a bit as a excuse to familiarise myself with the problem.

In order to include the loaders into my project I had to duplicate the files and convert this pattern:

THREE.XXXLoader = function ( manager ) {
    ...
};

To this:

import * as THREE from '../three.modules.js';

function XXXLoader( manager ) {
    ...
}

export { XXXLoader }

I still don't know what the solution is though.

[ghost]

@backspaces The problem is not that we're not trying to be forward-compatible, it's actually about the implications for backward-compatibility: Imagine a newcomer/noob scenario; what's he supposed to do when all of the sudden, he can't just put <script> tags in his HTML anymore? Try and explain to him in one sentence why he would need a compiler for ES, because I'm sure he'll just reply with something like "Why? It's just a scripting language, I didn't have to compile it before. This is stupid!"

I'm not saying that the global approach should be supported forever, but making this change now would exclude beginners until <script type="module"> is supported in most browsers. They simply couldn't use this project anymore, and all tutorials out there would be null and void.

Yes, it is a big hassle to work with Three in a compiler-based setup, but both SystemJS as well as Webpack offer ways to link globals in packages, so at least these users aren't completely excluded.

For reference, here's a list of bugtracker issues for all major browsers that shows how far the implementation of <script type="module"> is:

• Blink: #1569
• Gecko: #568953, #1240072
• Webkit: #147340, #148897 (Modules already implemented in JavaScriptCore, the rest is just a matter of exposing them to the web)
• EdgeHTML: Available in Edge preview, Build Number 14342+

My personal guess is that this will probably ship in early 2017.

[backspaces]

OK, I've sent Guy an email and posted to the gitter for System.js. I'll look for more but ..

I've tried several experiments with THREE, OrbitControls and a simple scene.

Without modifying any three.js code:

• THREE imports nicely, and is local, not modifying the global/window space.
• Similarly OrbitControls can be imported using an "empty import": import 'etc/OrbitControls.js' (empty imports do not return a module, simply run the code for side effects) BUT it fails trying to install in THREE .. i.e. THREE.Foo = {...} fails due to an import being invariant.

But there is a semi solution. Use only one <script> tag for THREE, and imports for "extensions". This works because the <script> simply places THREE as an Object in window. This allows the empty import to work as expected.

This actually is useful for our project because we want many modules to have their own "private" uses of THREE extensions w/o adding more html script tags. Pretty clean.

With Modifying OrbitControls

• Adding an import of THREE to OrbitControls.js imports correctly, but fails on the first assign to THREE, due to invariance.
• Removing the 3 THREE assignments, and exporting via: export default OrbitControls works but again, cannot be assigned to THREE, so it's its own module/name OrbitControls.
• In my simple test, I added import OrbitControls from 'lib/ModOrbitControls.js' .. the modified OrbitControls. Works fine but unfortunately does not have the THREE prefix. I.e. used like this const controls = new OrbitControls(camera, renderer.domElement)

So in sum, without changes to the three code base, you can use a script tag for THREE, yet have modules use empty imports that will modify the global THREE object.

With changes, it all seems to work with the one problem that the extensions can not be in the THREE object which is now an invariant module but no longer in the global scope.

Whew! But this is a pretty small sample, I don't know what other parts of THREE will need work.

[backspaces]

After trying several ways to use three.js with es6 modules, these are the results:

1 - Legacy: Use <scripts> tags for three.js and all three.js plugins. By plugin, I mean scripts that have THREE.OrbitControls = {...} assignments. This does not scale well for a team, the html is in one repo, the JS code in others. Any developer choosing a new navigation controller, for example, would have to modify all html files using the code.

2 - OneScript: Use a script tag only for three.js, and use "empty modules", import 'etc/OrbitControls.js' for plugins. These modules don't set a module name, only have "side effects". This works due to window.THREE being universally visible and an ordinary object, thus can be extended by the plugin. Slightly risky but works with system.js which seems to be standards compliant. And keeps dev dependencies local w/o editing html files.

3 - ThreeModule: Three already works as a "global" system.js module. I.e. import * as THREE from 'etc/three.min.js' treats three.js as an es6 module: local scope, no global setting, no script tags. It woks perfectly for scripts using no THREE plugins. Unfortunately, this will not work with importing legacy plugins for two reasons: THREE will not be visible within the plugin (not global, no local es6 import of three.js), and any assignment to THREE will fail due to it now being an immutable module. The "empty module" stunt now will not work.

4 - ModulePlugins: Above I did not modify the plugin. Converting OrbitControls to a module is trivial: Use the three import above, replace the 4 THREE.OrbitControls = {...} assignments to OrbitControls = {...}, and export OrbitControls. This works well. The only difficulty for users is the use of OrbitControls rather than THREE.OrbitControls but being in modules, this is easily explained to module users. If possible, I'd look to see how to convert ModulePlugins back to the current form, keeping a single code base.

One question I need an answer to: How many plugins are there in the repo? I.e. how many files would need to be converted to ModulePlugins?

If only those in examples/js/controls/ and follow OrbitControls' style, I'd be tempted to create ModulePlugins, hopefully automatically. But if not, probably use the OneScript approach.

[ghost]

@backspaces Sorry, I don't understand what you're trying to achieve.

If you're using Webpack, just import the examples by setting up the imports-loader plugin. If you're using SystemJS, read this reply; you can link modules to globals there, too.

You don't need any <script> tags for three.js-related stuff, nor do you need to modify the source in order to use them as ES2015 modules. All of the steps needed have been discussed or pointed to already. It's just a matter of (painful) module bundler configuration, but that's the way the transition is carried out at the moment, until better solutions allow us to re-write the examples in ES2015. (And one of the obvious solutions would be <script type="module">.)

You don't need to touch any of this project's files nor introduce globals in the browser.

[backspaces]

@kdex: My goal is to use three.js and all related files (like the controls/ files) as es6 modules using only system.js and the babel module transform.

Is this the link you mentioned? https://github.com/systemjs/systemjs/issues/666

[ghost]

@backspaces Perfect, because in that case, your module bundler can already do that for you. And yes, that's the thread I was referring to.

[backspaces]

I don't use a bundler (rollup?, webpack?) just the simplest possible use of system.js via babel and its system.js transform .. nothing else.

All my modules are separate files so only the modules that are needed are loaded. Are bundlers needed as long as I'm on a modern browser and have no need to support "legacy" systems?

I hope I'm not being an idiot here, possibly not understanding the system.js config .. which is definitely my next homework, thanks .. I really appreciate the help!

[ghost]

I'm not entirely sure if this setup will do; you'd have to set up SystemJS with a config.js (I think you'll need jspm for that, and drop the transform).

You do need a way to run ES2015 modules in today's browsers (not only legacy browsers). None of the browsers supports ES2015 modules yet. So, there needs to be a step in your build pipeline that removes all the imports and generates equivalent ES bundles. AFAIK, people nowadays use webpack or jspm (the latter is based on SystemJS) to do that.

In their configuration files, you will have the option to set up their behavior for scenarios where something you're trying to import depends on an injected global. (This was discussed earlier. Parts of three.js depend on a global named THREE, which you would have to set up there.)

I'd suggest extending your build pipeline to use babel and a bundler of your choice. Beware that setting this up might be a painful process; that's essentially why this thread is here.

Anyway, let's try and not get sidetracked in here. Still not a support thread! 😄 Good luck!

[JamesHagerman]

I ran into this over the weekend after realizing some coworkers were also running into this. We're using Webpack.

Is this something rollup might be able to handle for us? Perhaps leave the normal build process alone but provide a custom rollup config that can export each of the examples/js/ files? Then, those of us that would like to use a loader can just stay out of the way.

Unfortunately it seems as though rollup can't handle multiple exports by default; it does export default but won't allow anything else to be exported :frowning_face:

I'll dig into this a bit tonight. I'm new to loaders but I can't believe there isn't some clean way to pull this off.

[danrossi]

You can make a custom rollup config and build area. The main three.js file just needs to point to the right import paths outside the source directory.

The example files I am forced right now to append the ones I need, I included code above.

[backspaces]

What is three.modules.js? Is it somehow better than three.js for importing via system.js?

[looeee]

@backspaces

src/Three.js and all the imported files from that folder, are converted by Rollup into a single file.

Rollup generates two files.

The first, build/three.js, is in 'umd' (universal module definition) format - that is, it's compatible with ES5 environments.

UMD format allows the files to work as (from here):

• an AMD module to work with module loaders like RequireJS
• CommonJS, suitable for Node and Browserify/Webpack
• an 'iife' module, AKA a self-executing function, suitable for inclusion as a Githubissues.
• Githubissues is a development platform for aggregating issues.