riot tag based coverage report incorrectness

[inancgumus]

• LCOV reporting generates wrong coverage report on source code lines
• While, HTML reporting works perfectly fine

You can see the difference in below screenshots.

How is it possible that LCOV reporting gets its data from unprocessed src/hello.tag file while HTML reporting gets its data from processed src/hello.tag file? What is the difference? I've read their source codes but I couldn't find anything there.

Reproduce:

Example project:

git clone https://github.com/vitogit/tdd-mocha-chai-riot

Karma Config (updated):

module.exports = function(config) {
  config.set({
    basePath: '',
    frameworks: ['mocha','chai','riot'],
    plugins: [
      'karma-mocha',
      'karma-mocha-reporter',
      'karma-phantomjs-launcher',
      'karma-chai',
      'karma-riot',
      'karma-coverage'
    ],
    files: [
      'node_modules/chai/chai.js',
      'src/**/*.tag',
      'test/**/*.js'
    ],
    preprocessors: {
      '**/*.tag': ['riot', 'coverage']
    },
    coverageReporter: {
      dir: 'coverage/',
      reporters: [
        // works correctly
        { type: 'html', subdir: 'report-html' },

        // does not work correctly -- gets its input from unprocessed riot tag file
        { type: 'lcovonly', subdir: '.', file: 'lcov.info' }
      ]
    },
    browsers: ['PhantomJS'],
    reporters: ['mocha', 'coverage'],
    failOnEmptyTestSuite: false,
    autoWatch: true
  })
}

How to run?

npm i && npm i karma-coverage --save-dev
karma start
cat coverage/lcov.info

You can also see html based coverage reports inside coverage directory's html coverage reports.

LCOV is incorrect:

HTML is correct:

[inancgumus]

By the way, actually, what I really wonder is why html coverage output is true while lcov coverage output is not?

[weblogixx]

I have the same problem. Lcov-Reporter shows incorrect line numbers, which makes me guess it really works on the already transpiled files (in this case react, but I do not think that this matters).

Output for text, html and cobertura seem to be fine... The main problem is that I cannot get other plugins to work in SonarQube, so I have to use lcov...

Is there any news on this problem?

[inancgumus]

@weblogixx Karma made everything complex. So, I've switched to more basic approach: Run mocha and istanbul directly from CLI through npm run. This way, what happens is clear and LCOV also works correctly, because I don't need to deal with karma plugins etc. I'm directly dealing with the core components directly.

[weblogixx]

@deeperx Thank you for your answer. This is interesting, it seems to be more like a karma coverage reporter problem then? Unfortunately, I am not able to change my setup from karma to pure mocha (I know it technically is possible, I already did it in one or two projects I am currently working on, but I have some projects that really depend on karma :().

[inancgumus]

Yes, it seems like that. It made so mad that I'd completely reconfigured to pure CLI based tools :)

Inanc Gumus

21 Tem 2016 tarihinde 08:23 saatinde, Chris notifications@github.com şunları yazdı:

@deeperx Thank you for your answer. This is interesting, it seems to be more like a karma coverage reporter problem then? Unfortunately, I am not able to change my setup from karma to pure mocha (I know it technically is possible, I already did it in one or two projects I am currently working on, but I have some projects that really depend on karma :()�.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.