broccolijs / broccoli

Browser compilation library – an asset pipeline for applications that run in the browser
https://broccoli.build
MIT License
3.33k stars 217 forks source link

[FEATURE] Input node change tracking #419

Closed thoov closed 5 years ago

thoov commented 5 years ago

Background

This PR adds opt-in input node change tracking. If enabled, Broccoli will pass an object as the first argument into the build method of plugins when invoking them. We are passing an object as it will give us greater flexibility to add additional properties later.

Implementation

The type of this object is:

interface BuildObject {
  changedNodes: boolean[]
}

By default this will not be passed into the build method. To enable this feature, plugins must pass trackInputChanges:

class MyPlugin extends Plugin {
  constructor(inputNodes) {
    super(inputNodes, {
      trackInputChanges: true
    });
  }
}

Use Case

The primary use case for this feature is to enable "selective building". An example of this is if your plugin takes several inputs and performs some action on each inputPath. This can become inefficient with either a large number of inputs or if your action is expensive. Currently plugin authors have to create their own diffing solution which generally is inefficient due to having to touch the filesystem. This feature leverages the revision tracking system and thus can efficiently inform plugins on which node has changed. A concrete example of where it will be used is in embroider to allow addons to be rebuilt: https://github.com/embroider-build/embroider/pull/297.

Example

const fs = require("fs");
const Plugin = require("broccoli-plugin");
const { WatchedDir } = require("broccoli-source");

class MyPlugin extends Plugin {
  constructor(inputNodes) {
    super(inputNodes, {
      trackInputChanges: true
    });
  }

  build({ changedNodes }) {
    this.inputPaths.forEach((path, idx) => {
        if (changedNodes[idx]) {
          const input = fs.readFileSync(`${this.inputPaths[idx]}/foo.txt`);
          fs.writeFileSync(`${this.outputPath}/${idx}-foo.txt`, input);
        }
    });
  }
}

export default () => {
  let app = new WatchedDir("app");
  let foo = new WatchedDir("foo");

  return new MyPlugin([app, foo]);
};
thoov commented 5 years ago

@oligriffiths could you take a look at this

thoov commented 5 years ago

@rwjblue The goal is that if you are a plugin that does something on each one of your inputNodes there isn't a great way to know which one of them changed on a rebuild. This exposes the ability to easily know which one of your inputs have changed instead of having to come up with a diffing algo.

oligriffiths commented 5 years ago

Seems the right direction. I wonder if it would be better or not for the object passed to expose as hasChanged(inputPath) method or not. Throwing it out as a suggestion

thoov commented 5 years ago

@oligriffiths @rwjblue @krisselden Could you guys review this again. I made it opt-in (per Rob's suggestion), added some tests, and have updated the description.