Reports don't show correct lines covered

[scottohara]

I'm attempting to use babel-istanbul with karma-coverage and source/test code compiled with babel 6:

babel-core@6.5.1
karma-coverage@0.5.3
babel-istanbul@0.6.1

I'm getting the correct coverage results; but the HTML reports don't seem to reflect the actual lines that were covered/uncovered.

Here's a simple contrived example. First we have simple class Foo, with two methods bar() and baz():

class Foo {
  bar() {
    return true;
  }

  baz() {
    return false;
  }
}

A test that has 100% coverage for this class is:

describe("Foo", () => {
  let foo;

  beforeEach(() => foo = new Foo());

  describe("bar", () => {
    it("should be true", () => foo.bar().should.be.true);
  });

  describe("baz", () => {
    it("should be false", () => foo.baz().should.be.false);
  });
});

The relevant parts of karma.conf.js for this are below. Basically, I'm using karma-babel-preprocessor to transpile the source (using the es2015 preset); then karma-coverage with instrumentation by babel-istanbul:

  preprocessors: {
    "**/foo*.js": ["babel", "coverage"]
  },
  babelPreprocessor: {
    options: {
      presets: ["es2015"],
      sourceMaps: "inline"
    }
  },
  reporters: ["mocha", "coverage"],
  coverageReporter: {
    instrumenters: {
      "babel-istanbul": require("babel-istanbul")
    },
    instrumenter: {
      "**/foo*.js": "babel-istanbul"
    },
    reporters: [
      {type: "html", dir: "coverage"},
    ],
  },

The command to run the test suite is simply karma start.

The result is 100% coverage as expected; and the HTML report looks like this:

Now if I remove the test that invokes Foo.baz() (ie. delete the whole describe("baz", ...) section) and run again, the coverage drops (as expected) to 96%.

But looking at the HTML report, it doesn't really indicate that it is the baz() function that was not covered:

Is this working as intended?

[jeffrifwald]

This module is definitely designed to help you view coverage on your source code, not on compiled code like you have shown. I'm not sure how you have set up karma-coverage, but I think your issue is related to how it is compiling.

I believe karma-coverage is pre-compiling your code and your tests are being run from the pre-compiled code. babel-istanbul is never even aware of the source code, so it can't show you coverage on it. babel-istanbul needs to be pointed to the original source code so it can compile it, generate the source map, and do its thing.

I'm actually not familiar with how karma-coverage is set up internally, but basically you'll need to figure out how to stop it from sending compiled files to babel-istanbul.

[scottohara]

I see, thanks for the explanation @jmcriffey.

With so many tools now wanting to do their own babel compilation (rather than just relying on the compiliation to have already occurred once somewhere upstream in the tool chain, and use the supplied source maps to map back to the original source), I think in my case I'll just wait for the official istanbul repo to finalise it's sourcemap support (https://github.com/gotwarlost/istanbul/issues/212).

Otherwise, I'll see if the karma-coverage folks have any suggestions on how to have the instrumentation done on the precompiled source. Perhaps it is as simple as swapping the order of the karma preprocessors, from:

preprocessors: {
    "**/foo*.js": ["babel", "coverage"]
  },

to:

preprocessors: {
    "**/foo*.js": ["coverage", "babel"]
  },

[scottohara]

Just a follow up, in case anyone else is looking at getting babel-istanbul and karma-coverage to play nicely....

I removed the karma-babel-preprocessor from my config, so that babel-istanbul sees the uncompiled es2015 source (and let it handle the compilation/source map generation instead).

The karma.conf.js file ends up looking like this:

  preprocessors: {
    "**/foo*.js": ["coverage"]
  },
  reporters: ["coverage"],
  coverageReporter: {
    instrumenters: {
      "babel-istanbul": require("babel-istanbul")
    },
    instrumenter: {
      "**/foo*.js": "babel-istanbul"
    },
    reporters: [
      {type: "html", dir: "coverage"},
    ]
  },

With that in place, I was able to get the coverage reports to correctly show the original es2015 source and the correct lines being covered.

However...

This was without any .babelrc file; which made me think: babel6 doesn't run any transforms by default; typically it requires presets: ["es2015"].... so is my code actually being transformed, or is it working simply because Chrome supports es2015 classes by default?

So I added the following .babelrc to the root of my project:

{
  "presets": ["es2015"],
  "sourceMaps": "inline"
}

With this in place, I now get a bunch of errors:

15 03 2016 11:08:49.144:ERROR [preprocessor.coverage]: Cannot read property 'sections' of null
15 03 2016 11:08:49.147:ERROR [karma]: [TypeError: Not a string or buffer]

Something (not yet sure what, whether its karma-coverage or babel-istanbul) doesn't like the .babelrc settings.

Will continue to troubleshoot and see if I can get anywhere.

[mmahalwy]

@scottohara any luck withthis?

[scottohara]

@mmahalwy Yes, the problem was the "sourceMaps": "inline" in my .babelrc.

It would seem that removing this fixed the issue (and babel-istanbul doesn't seem to need it, and still generates source maps inline without it).

[adimittal]

Toggle --source-map option to true while running angular tests with code coverage report gen ng test --source-map=true --code-coverage