systemjs / builder

SystemJS build tool
MIT License
465 stars 122 forks source link

Avoid regenerating bundles that haven't changed #839

Closed IceCreamYou closed 6 years ago

IceCreamYou commented 6 years ago

I have a custom Grunt task, grunt bundle, which uses SystemJS Builder to create multiple bundles. I would like to reduce the amount of time it takes to run that task. To that end, I am thinking about how to avoid regenerating bundles when the underlying modules haven't changed, since often only one bundle will need to be rebuilt.

One way to do this could look something like:

  1. After generating a new bundle, write the entry point expression to disk somewhere
  2. Before generating new bundles:
    1. Read the bundles property from the SystemJS config and the map of bundles to entry point expressions from wherever we saved that
    2. Stat the bundle files and the modules for each bundle
    3. If the potential new bundle would have the same name and entry points as an existing one, check whether any of the existing bundle's modules have an mtime greater than the existing bundle file. If not, don't bother regenerating it since nothing will have changed.

This seems possible to do by extending my custom Grunt task, though it's not trivial. Does this seem like a sensible thing to do, or do any risks jump out?

One potential addition could be to use the builder's getCache() and setCache() methods to write the builder cache to a file after each task run and load it up before the next task run. However, getCache() and setCache() are not documented in api.md. Are these functions, including the structure of the cache objects, considered stable public APIs? Also, the serialized output of getCache will be quite large. Does it seem likely that the cost of reading and parsing the persisted cache will outweigh the benefits of keeping it around, considering that much of it will normally need to be invalidated anyway?

Finally, am I missing any better approaches?

Thanks in advance for thoughts/help here.

IceCreamYou commented 6 years ago

I ended up implementing something along these lines today and it brought typical bundle times down 20-30X.

I am still interested in whether getCache() and setCache() are considered stable public APIs, but I will close this issue since my motivating problem is solved for the moment.