Async module loading / bundle splitting

[yoshuawuyts]

As per https://github.com/browserify/discuss/issues/5:

Probably the largest unsolved problem in the ecosystem is asynchronous module loading / splitting files into multiple small files. factor-bundle allows extracting shared code between bundles. The bundles then rely on being loaded in the right order. partition-bundle is similar, but allows bundles to be loaded with a loadJS() function.

Input on the topic would be greatly appreciated!

cc/ @kristoferjoseph @brianleroux

[terinjokes]

From our experience here at Cloudflare, getting factor-bundle to work reasonably well was a real hassle. It definitely felt bolted on, rather than something that's part of the Browserify project.

[yoshuawuyts]

@terinjokes in your opinion, what could be done to improve this? Any particular parts of integrating it that didn't feel right?

[kristoferjoseph]

Description

Hello all, I've been researching code-splitting with browserify for a bit now and can share what success criteria looks like for me.

I can break down what would be ideal — with some explanation as to why — then list out my experiences with using existing libraries and what deficiencies i ran into using them.

This is a fairly complex issue so please bear with me, also please by no means take this as gospel. This is just meant to add some ideas about success criteria based on using this approach "in anger" while building a real app.

success criteria

• main JavaScript file
  • this file should bundle shared modules from other entry files as well as a way to load the other un-shared modules
• entries
  • these are pages/screens/sheets/modals/what have you, usually accessed by a url in the browser. These are distinct starting points for a flow through an app or website. Supplying these allows us to split out shared code into a common bundle as well as only load what is needed for a given url.
• loader
  • this is used both during local development as well as once deployed to load modules by appending a script tag to the html head or streaming modules.
  • should be able to be configured with a url to append to a script source i.e. localhost:3000 ( local ) or https://some.s3.bucket ( deployed to staging and / or production )
  • should keep track of what modules have been added already. Meaning if you are on page a you can add <script src="https://some.s3.bucket/main.js"> and </script><script src="https://some.s3.bucket/a.js"></script> during static render and not re-append the a.js file on load.
  • this loader should also be able to append multiple external module scripts as needed. For instance if you have a large shared dependency like moment.js that you only use on three of the 20 urls you should be able to externalize this file at bundle time and only append it as well as the page file as needed.
• CSS
  • it would be very nice to be able to export a css file per entry point that is loaded only for the pages that need it. This could easily be a sheetify plugin me thinks.
• configuration
  • this could take a bit of configuring for any level of complex app. this could live in a key of the projects package.json ( which is my ideal approach ) or another external configuration file, or in a build script.

partition-bundle discovery

partition-bundle on github Here is an example build file that uses partition-bundle

For me partition bundle works pretty well. It satisfies almost all of my success criteria save for splitting out page specific css files, but this could be a sheetify plugin concern.

The example build file shows passing the main file, entry files as well as a url for local and deployed loading of files with the bundled loadjs

factor-bundle discovery

factor-bundle on github factor-bundle is really very powerful for factoring out common code. With the correct combination of exclude and require settings in the browserify arguments you could achieve similar results as partition-bundle.

However you would need to come up with your own loader which might be a good idea in that it separates concerns, but it does make it a non-viable option out of the box.

Another issue i ran into was it didn't put files into a main entry file with multiple other entry points. In my experience it only separated out shared modules into a common bundle. For me it would be better to have one file that manages common code as well as loading other externalized modules.

Anyways, there are some thoughts, hopefully they help.