Moving to a modular architecture

[kumavis]

Browserify Moving to this architecture has advantages and disadvantages. Please add your thoughts.

Note: this does not require three.js consumers to use browserify.

[rasteiner]

You normally use watchify during development with livereload. That one builds the bundle incrementally.

[nucleartide]

watchify doesn't work for me. When I change a file and save it, watchify and beefy's livereload serve up the older/cached version. I have no idea why this happens. Thankfully, browserify already works pretty well.

@ChiChou Pass in --noparse=three to browserify. This takes the browserify bundle step from 1000ms down to 500ms on my machine, which is decent enough for an instant feedback feel.

[kumavis]

@rasteiner I want to thank you again for your informal research into three.js inter-dependencies. While that massive list of deps is some ugly looking code, really that ugliness is present as is, just invisible. Browserify's strength is it requiring us to air our dirty laundry and pursue less tangled systems.

[kumavis]

There are a lot of places in Three.js where we take in some object, perceive its type, and perform different tasks based on that type. In most of those cases, that type-dependant code could be moved to the type itself, and we would not have to have an understanding of all the possible types that we're operating on.

The following is an abridged example from WebGLRenderer:

if ( texture instanceof THREE.DataTexture ) {
  // ...
} else if ( texture instanceof THREE.CompressedTexture ) {
  // ...
} else { // regular Texture (image, video, canvas)
  // ...
}

should be more of the form

texture.processTexImage( _gl, mipmaps, otherData )

Let the type determine how to handle itself. This also allows the library consumer to use novel Texture types that we had not thought of. This structure should reduce interdependency.

[ghost]

I think moving to a browserify architecture is definitely the way to go. The UMD build will make consuming THREE.js easier. It will also allow us to split the WebGLRenderer into multiple files, because right now it looks rather monolithic and scary.

I have started a branch where I am currently working on moving it over here: https://github.com/coballast/three.js/tree/browserify-build-system

Please let me know what you think.

[kumavis]

Here is @coballast's changes.

It looks like you're taking the automated conversion approach with your browserifyify.js file, which I think is the right way to go.

One thing that we all haven't discussed much yet is how best to transition this large, ever-changing library over to browserify. You can make the changes and then open a PR, but it will immediately be out of date. Thats whats compelling about the automated approach.

If we can:

• provide a three.js src conversion script (like your browserifyify.js)
• provide a document that explains how the conversion process works
• provide a document that explains how the new build system works
• run tests post conversion
• not include any changes to existing files that could lead to a merge conflict

...then we can turn this into a push-button conversion that will still work into the foreseeable future. that simplicity enables this dreamy notion of a fundamental architecture shift to such a large project to be accomplished when the ideological arguments win out.

[kumavis]

@coballast to that end, I would remove the changes to src/Three.js if it works just the same.

Note: not just revert, but remove those changes from the branch's history via a new branch or a force push

[kumavis]

@coballast I wonder if it would make more sense for the conversion utility to be not a fork of three.js, but an external utility that you point at the three.js development dir, and it converts the source files, adds a build script, and runs the tests.

[ghost]

@kumavis I agree with leaving the src directory alone. I think the thing to do is have the utility write a duplicate directory with the commonjs code, and we can test and run the browserify build from there to make sure the examples all work before we try doing anything major.

There is also an interesting opportunity here to write some static analysis stuff that will automatically check and enforce consistent style across the entire codebase.

[kumavis]

@coballast sounds great. There's a wealth of tools out there for automated code rewriting, e.g. escodegen. Need to make sure we're maintaining comments, etc. Want to start a threejs-conversion-utility repo?

[kumavis]

@coballast that said, maintaining a sharp focus for this utility is important

[ghost]

@kumavis Consider it done. I really want this to happen. This is only one of two projects I am working on at the moment.

[ghost]

@kumavis @mrdoob Some of the discussion here seems to be around the idea of splitting THREE into multiple separate modules that could presumably be installed via npm and then compiled with browserify. I am not outright against the idea, but I think that should be a separate issue. The only thing I am advocating at this point is using browserify to manage THREE's codebase internally. Get it moved over, and get it working the same way it always has for users, and then evaluate what makes sense.

[mrdoob]

I'll be curious to see what's the output of that utility ^^

[kumavis]

@coballast link a repo for us to track, even if its just empty at this point. We can build from there.

[ghost]

https://github.com/coballast/threejs-browserify-conversion-utility

The code is a mess, will clean up soon.

[kumavis]

here we go! :rocket:

[ghost]

I now have the utility in a state where it generates browserify src and will build it without problems. I will update the repo with instructions on how to do this yourself. At this point, the examples don't work. There are several issues that need to be dealt with to fix that. I will add them to the repo if anyone wants to roll up their sleeves and help out.

[kumavis]

@coballast yeah please file the issues as TODOs, and I or others will jump in as we can.

[ghost]

Serious problems have come up. See #6241

[kumavis]

Here is my analysis of what would need to happen in order for this to work: https://github.com/coballast/threejs-browserify-conversion-utility/issues/9#issuecomment-83147463

[spaesani]

browserify is at the very least transport redundant (conjestive) due to it's design. This makes it's use cost inflative(data plan anyone?) and slow.

A simple remedy to this is to seperate document from library code which would entail two client files and not one. This is common practice for client side js.

If at the outset browserify is faulty and itself needs to be fixed I can hardly see why it should even be considered to improve anything at all let alone something like threejs.

[rasteiner]

@spaesani Because the data for threejs has to be downloaded anyway. If we split threejs into smaller modules and let an automated build system cherry pick what it needs for a single app, actually most threejs apps out there would be lighter.

If for some reason you still want to separate "document from library code", you could still do this and use a pre-built version like we do now. You could even split your browserify-built app into separate modules by using the --standalone and --exclude flags.

Browserify is just a way to use a battle proven module definition API (CommonJS) on the browser. It would greatly simplify the development of threejs plugins and enhance code clarity and therefore productivity, it would allow us to integrate into a bigger ecosystem (npm) where the code is inherently maintained by more people while still maintaining integrity through the versioning system (think about the stackgl family), and it wouldn't even force people into CommonJS if they don't want it.

Of course there are downsides, but they're not the ones you've mentioned.

[spaesani]

three.js and three.min.js can be cached to save on transport (data) beit by a proxy,  the common mobile solution,  or a caching browser. The moment you cherry pick and aggregate the threejs code with document specific code the caching is not feasible. If browserify allows one to Githubissues.

Githubissues is a development platform for aggregating issues.