Add option to build with webpack instead of grunt-browserify

[cheapsteak]

@njam3 @mattdesl @mikkoh

Cost: TBD (respond below)

Benefit: TBD (respond below)

[cheapsteak]

Benefit:

• So much faster than grunt-browserify
• Easy code splitting (vendor.js, app.js)
• Hot module replacement (plug-and-play options for React and Vue)
• Dead code removal (tree shaking)

[mattdesl]

Cool. I will add some stuff as I go.

So much faster than grunt-browserify

We should compare against budo as we intended it to replace grunt-browserify #79.

[cheapsteak]

How does budo fare on the other points?

• Code splitting is useful for debugging build files so you don't have to load a 5mb js file if you want to look through the compiled code (and for instances where the chrome console doesn't use source maps)
• Hot module replacement is great for live editing without losing state

But why not both?

Ask during setup whether the project should use budo, webpack, or grunt-browserify for building?

[mattdesl]

Ask during setup whether the project should use budo, webpack, or grunt-browserify for building?

grunt-browserify should probably not even be an option for development. It is a nightmare to set up with features like LiveReload and the development experience is poor (like watchify speed).

The point of the generator is to keep all our projects consistent, so that any developer can jump on a project and have a good idea of how things work, how to develop with it, and so forth. Adding new and radically different build tools as options makes it more difficult to maintain the generator and also leads to less consistency across our projects.

I think it will be easier to strip all the grunt/browserify stuff out of the generator completely, and re-work it with webpack. With the "webpack approach" you rarely need a complex Gulp/Gruntfile since Webpack intends to solve so much of the build step. Of course, we would only do this after evaluating webpack on a project to know what features are needed, and after everybody on the team is OK with the switch.

[cheapsteak]

Agreed @BrendanNeufeld @jmckinnell @mikkoh wanna be the guinea pig pilot project for webpack?

[jmckinnell]

woah now, he said "grunt-browserify", "gulp-browserify" is still very viable.

(sidenote, sorry for the close, misclicked)

[njam3]

Its already on the roadmap to remove grunt. https://github.com/Jam3/generator-jam3/issues/126, hoping we won't have to use gulp either, and just use everything through npm scripts.

[cheapsteak]

Oh yeah @njam3 I forgot you have Mural coming up soon, any thoughts on what you'd be using for compilation?

@jmckinnell gulp-browserify is actually just browserify, no plugins required!

There is no shortage of alternatives that are way way better than grunt-browserify, but it seems that new projects are stuck with grunt-browserify until .. there's kind of a chicken and egg problem, can't add stuff to generator unless people have used it (not even to test it out as an option during setup), but projects are all started using generator, and there's a high barrier for new projects trying out new things. It's been about a year since people have been whining about how slow grunt is, and yet it seems we're still a long way away from change

[mattdesl]

It should be pretty easy to change grunt-browserify to budo. The fact that it hasn't happened yet shows how little time we have to devote to the generator. This change is very small compared to the refactor that would be needed to have the generator operate on webpack instead of Grunt/browserify.

I have been testing webpack today and how it relates to modules specifically, not apps. Basically, exploring how it would change the development process for modules like these:

• three-bmfont-text
• google-panorama-by-location
• mesh-heightmap-contours

The main problems I'm finding with building modules using webpack:

• Various npm modules (which work in browserify) will not work in webpack without configuration, such as:
  • Dependencies using require('./blah.json')
  • Dependencies using require('fs') like tape
  • Your npm dependencies may add these in a patch version, causing your webpack demo to break on its next install
• Unit testing in the browser is more difficult
  • Developing unit tests for WebGL/WebAudio/etc needs to use webpack-dev-server instead of purpose-built tools like hihat and tap-dev-tool
  • Since webpack doesn't write to stdout, smoke tests with smokestack, testling and tape-run need to use a temp file
• Webpack CLI writes to stdout which can affect some unix tools, no --silent option #490 (minor irk)
• webpack-dev-server
  • Does not auto-generate an index.html (minor irk; needs a plugin)
  • Save + Reload cycle is slightly slower than with budo (minor irk)
  • Does not reload on Ctrl + S when file is unchanged (minor irk)
  • Lacks some features of budo like DOM error reporting
• Issues with transforms:
  • Webpack has less emphasis on Node compatibility, the "isomorphic" brfs approach doesn't work
  • Some tooling for browserify doesn't translate into Webpack loaders; e.g. per-file transforms for glslify-loader

Pros:

• Webpack2 will feature tree shaking; this is an awesome feature
• Code splitting
  • Browserify has factor-bundle, parition-bundle, but Webpack's is more complete and better tested.
  • However, this is overkill for the vast majority of isolated module demos.
• Hot Module Replacement
  • Browserify has browserify-hmr and other plugins, but Webpack's is more feature-rich and better tested.
  • If your module is not specific to Vue, React, or something that already has HMR boilerplate, you probably won't bother using this.

Overall, I'm not really convinced to switch to webpack for my modules since it is a step backwards in terms of productivity, provides less Node compatibility than browserify (i.e. npm/node dependencies may break your demo), and introduces more configuration to maintain and develop per-module. However, I can definitely see its values on project like Code Splitting, HMR, etc. I have yet to evaluate it for this purpose.

[njam3]

Yeah, switching to budo should have already happened but my generator time got eaten up by projects. Realistically, we only discussed changing to npm scripts a little over a month ago. I hope to find time before Mural to at least get budo setup.

[mikkoh]

Maybe we can adopt some stuff from Vikings for the generator. I'm currently using budo for all dev

https://github.com/Jam3/prj-vikings/blob/dev/package.json#L50

The npm script isn't the prettiest can def be cleaned up. Agreed grunt-browserify is too slow. Before it was ok but now transforms like babelify have become the name of the game and I think this is def slowing stuff down.

For code splitting there are many options with browserify eg factor-bundle https://www.npmjs.com/package/factor-bundle

Browserify External Bundles https://github.com/Jam3/external-bundle

I've known about these things for awhile the hard thing is where do you split the code? This is a discussion we were having with the tech directors yesterday at a minimum our projects should start doing the following: preloader.js loads bundle.js

preloader.js contains only the bare minimum required to preload everything else. This will eliminate having to no preloader when loading the main.js

Beyond splitting code after that it's difficult. It's a consideration I've made for bigwheel but had no time to actually think about implementation: https://github.com/bigwheel-framework/bigwheel/issues/14

Basically what I'm saying is sure webpack can split code but what does that actually mean in practice?

[cheapsteak]

sure webpack can split code but what does that actually mean in practice?

I'm not 100% sure I understand your question, let me know if I'm off -

So far I think we'd want preload.js, vendor.js, and app.js

Or are you talking about lazy loading each section?

I'm not convinced that we'd get much (if any) performance benefits from further splitting up of sections. Is our JavaScript really that big after you split out the vendor files? Each section would be maybe 10kb post-compress. Seems like a feature only useful for mega-apps and would be a premature optimization for marketing sites.

That said, this is how webpack handles it: https://webpack.github.io/docs/code-splitting.html

[mikkoh]

@cheapsteak are you thinking that vendor.js would contain all required modules etc?

[cheapsteak]

@mikkoh yes

Having a separate vendor.js is useful for debugging build files so you don't have to load a 5mb js file if you want to look through the compiled code of your app (and for instances where the console doesn't use source maps and gives errors at bundle.js line: 14336)

Additionally for any long projects with several launches throughout, vendor.js (probably the largest chunk) can be aggressively cached

[njam3]

I'm going to close this for now, but please reopen once webpack has been evaluated.