theintern / intern

A next-generation code testing stack for JavaScript.
https://theintern.io/
Other
4.36k stars 311 forks source link

Update Intern's configuration system #1109

Closed jason0x43 closed 4 years ago

jason0x43 commented 4 years ago

Intern's usage over the last couple of years has exposed some areas for improvement in Intern's configuration system.

Problems

The main problems are:

  1. The config-processing logic is spread out and can be hard to follow
  2. The merging logic is inconsistent (#877)
  3. The self tests aren't fully exercising the merge logic
  4. Some of the default options aren't applied consistently, leading to issues like #877, where attempting to add a reporter to the default simply replaces the default.

Background

The current process is split between core/lib/<env>/util#getConfig, core/common/util#loadConfig, and core/lib/executors/<executor>#_resolveConfig.

  1. Load a config file (the default or one given as a command line arg or query arg) - getConfig
  2. Process the config file, normalizing config values and processing merge directives - loadConfig
  3. Apply command line or query arguments - loadConfig
  4. "Resolve" the config (expand globs) - _resolveConfig. During this process, defaults stored in the executors are applied.

The config file format is JSON+comments.

The config file format is intentionally a bit looser than the internal config format for ease of use. For example, a config file can use a string for the suites property, but in a Config structure the suites property is always an array of strings.

Various parts of the config file may be merged during processing:

One substitution is currently performed: The string {pwd} in environment descriptors will be expanded to the current working directory at runtime.

Desired outcomes

Externalize the defaults

Apply the default config options currently setup in the executor constructors in the main config handling logic. The order should be:

  1. Default config
  2. Loaded config
  3. Environment variables (INTERN_ARGS)
  4. CLI / query args

This will require moving the default options into the config logic, or into separate environment-specific files.

Merging should always be explicit.

Given this:

{
  "suites": [ "a.ts", "b.ts" ],
  "node": { "suites": [ "c.ts" ] }
}

only c.ts should be used in a node environment. To get the existing behavior, use:

{
  "suite": [ "a.ts", "b.ts" ],
  "node": { "suites+": [ "c.ts" ] }
}

Merging should always be shallow

Merging should be shallow (it is now). Users can use '+' suffixes to explicitly merge subproperties.

Improved test coverage