v2 Idea Collection

[mitchellsimoens]

I'll do my best keeping a running list of what we want to add, change, remove from v1 in a v2 release. We are wanting to collect what people love about gimbal, what they dislike, what they hate, what they want from a tool like gimbal. There is no piece of feedback that we aren't wanting regardless if it's good or bad feedback. The more you give us, the more we can do to address them!

• [x] Run gimbal without the static server (against remote sites)
• [x] Look at the format of the config file
• [x] Ditch commander with something more lightweight and appropriate
• [x] Allow plugins to add CI/VCS
• [ ] Run in a test runner
• [ ] Think of a threshold replacement/addition
• [x] Find the build dir by default
• [x] Expose an api to plugins

[mitchellsimoens]

v1 uses a static server to host the build of the application. This is a great option till you get into more complex scenarios. Client project I'm on right now has some calls to the server made at the start of an application but these will fail because there is no server. I need a way for my CI process to startup the backend and frontend and gimbal be able to run audits against a URL I pass to gimbal.

In the same vein, it'd be great if the browser audits could be run on any remote web site. Not saying this is how it should work but running gimbal --url github.com should run browser audits against github.com. There are some audits that wouldn't run like the size and source map explorer since they are run against local files and that's fine.

[mitchellsimoens]

I love the config file but as things have changed, the format of the config file is weird and should be looked at.

For example, the builtin audits, to set thresholds you put them in the configs object but then you can set which audits you want in the separate audits array. This is an example:

audits:
  - size

configs:
  size:
    thresholds:
      foo.js: 1 KB

This isn't necessarily bad but when we added plugins, you set the threshold for that plugin within that plugin object like so:

audits:
  - axe

plugins:
  - plugin: '@modus/gimbal-plugin-axe'
    thresholds:
      bypass: serious

Once again, this isn't bad but it made me think, should we do the same thing we are doing in the plugins object in the audits array (or make audits an object):

audits:
  axe:
    thresholds:
      bypass: serious
  size:
    thresholds:
      foo.js: 1 KB

plugins can still have objects since they may have options (and the ones we supply do) but thresholds seem to go better in the audits and it's one place for them.

[mitchellsimoens]

We need to replace commander with something else like minimist since commander doesn't support async command actions. Plus we were executing other things outside of commander so it'd be great to just handle everything ourselves and with something more suited to be a cli option parser like minimist. Commander seems to overweight for our use now.

Need to also think that plugins can create their own command and options that we need to manage also.

[mitchellsimoens]

Should move to a manager for CI/VCS so that plugins could add their own in. It's fine to have Gimbal support the popular ones but there are some out there that we either don't have the time to add or don't know about and should let anyone add their own support in.

[mitchellsimoens]

We can programmatically call gimbal since it's just JavaScript in the end. It'd be great to allow gimbal to be executed from test runners, have each call cache their results and at the end reconcile all these results to give a final report. Most of this is already worked out including the reconciliation but caching results and running gimbal using a URL or a puppeteer page instance needs to be thought of.

This would get us around any auth issues since the tests themselves would handle that, we'd just run on a page provided by the test runner.

[mitchellsimoens]

Thresholds can be a pain. Need to come up with something that can fail a gimbal run but not be as big of a hassle as thresholds can be.

Maybe have ranges:

• good = 0-5KB
• warning = 5KB-20KB
• error = 20KB+

Having a warning range could alert something is growing but not fail the build.

[mitchellsimoens]

The build directory defaults to build which is great and easy to override but by default, let's have an array of common build directories and first one that actually exists is used... unless provided. So in code:

const possibleBuildDirs = ['build', 'dist'];
let buildDir = options.buildDir;

if (!buildDir) {
  buildDir = possibleBuildDirs.find(dir => fs.existSync(path.resolve(cwd, dir)));
}

This may make default running on different projects a bit more friendly (one less thing maybe).

[mitchellsimoens]

There was a bus concept in v1 so that the plugins installed locally can get same instances of things that gimbal (that may not be installed locally, could be globally) uses. Maybe instead, we have a full class instance that gets passed to the plugin to handle more.