Open boneskull opened 7 years ago
I've been working on this in the multi-coverage
branch.
Getting Coveralls to take reports from multiple build jobs in the same build turned out to be way easier than I thought. I wasted so many hours looking for a more complex solution because I didn't initially realize I could just apply COVERAGE=true
to the whole build. That alone should increase the coverage somewhat as it will show branches taken in different Node versions; would be nice to get more such branches covered though (Windows, whatever other terminals we're handling).
After that, I looked into adding coverage to the browser runs. Here's what I found out.
karma-coverage
can instrument the code and report on coverage for instrumented code, however...karma-browserify
doesn't look at the results of other preprocessors, therefore...browserify-istanbul
should be used to make karma-browserify
instrument the code, which can then be reported upon by karma-coverage
.browserify-istanbul
mysteriously doesn't run on anything going into the bundle that isn't in Karma's files
list and given ['browserify']
on Karma's preprocessors
list, so we have to add the relevant patterns to those lists even if they shouldn't be served directly.if (env.COVERAGE)
; then something changed and caused the current "bundle error" problems -- but I haven't been able to figure out what.browserify-istanbul
and karma-coverage
; this can be used to make browserify-istanbul
use the same version of Istanbul as karma-coverage
, or to make both use the same version of Istanbul as our Node CLI NYC. (This seems to work ok except for the above-mentioned issue, and removing it does not seem to resolve the above-mentioned issue.)Feel free to pull it down locally and experiment to confirm or deny any of the above and try to fix the bundle errors.
(I should add that the multi-coverage
branch is based off the CI fixes branch, for obvious reasons; the diff from master should get to be simpler once the CI fixes branch is merged, presumably...)
Found and fixed what I'd screwed up; that branch is working now (modulo Sauce flakes), so I've submitted PR #2886 for it.
We have coverage numbers, but only in Node.js. I'd like to see the coverage numbers against tests run with Karma.
Gathering coverage from Karma seems fraught with peril.
I would entertain other suggestions to improve how we run browser-based tests. My only requirement is that the tests run in a strict browser-like environment--
jsdom
shenanigans or Electron won't do.