theintern / intern

A next-generation code testing stack for JavaScript.
https://theintern.io/
Other
4.36k stars 310 forks source link

Allow unmet coverage to fail a test run #1098

Open jason0x43 opened 4 years ago

jason0x43 commented 4 years ago

Intern 4.x uses code coverage data to generate reports. However, this data could also be used to affect the output of a test run. A user should be able to set thresholds for the coverage data gathered using Istanbul (lines, statements, functions, branches), and if those thresholds aren't met, the test run should fail, even if the tests themselves succeed.

For the initial implementation, thresholds should apply only to summary data (as opposed to coverage for individual files). This will include coverage for all loaded modules, as well as any other modules included via the coverage config property.

If thresholds are set, they will be checked at the end of a test run. If no thresholds are set, coverage behavior is as it is now (for reporting purposes only).

The coverage config property should be updated to support setting thresholds, something like:

{
  "coverage": {
    "include": [
    ],
    "thresholds": {
    }
  }
}

Intern should continue supporting the simple coverage value, as well.