karma-runner / karma-coverage

A Karma plugin. Generate code coverage.
MIT License
770 stars 247 forks source link

HTML report and LCOV report discrepancy #390

Open raghanag opened 5 years ago

raghanag commented 5 years ago

Hi I am having a strange issue when generating code coverage with karma-coverage. My source files are written in typescript where as the test files are written in javascript with qunit being the test framework. When I run the karma start, all my tests are getting passed and the code coverage report being generated. But the istanbul report and the lcov.info which I am using sonarqube to view the code coverage are showing different reports.

For example, in html report by istanbul even though the source file is in ts, the code coverage shown in transpiled js file with all of the code is covered except the awaiter and generator functions which will be generated when we use Promises in typescript, since we are using the target as es-5.

The Istanbul report is showing as 65% lines coverage, but in Sonar the coverage is showing as 17%, please note that with the below coverageReporter configuration, it generates the lcov.info report which I am using it to publish it to sonar.

The Sonar dashboard is displaying the ts file as-is not in the transpiled/javascript code.

Here is the karma-config.

preprocessors: {
      'src/js/**/*.ts': ['typescript'],
      'src/js/**/*.scss': ['scss'],
    },

    typescriptPreprocessor: {
      // options passed to the typescript compiler
      options: {
        sourceMap: false, // (optional) Generates corresponding .map file.
        target: 'ES5', // (optional) Specify ECMAScript target version: 'ES3' (default), or 'ES5'
        module: 'amd', // (optional) Specify module code generation: 'commonjs' or 'amd'
        noImplicitAny: true, // (optional) Warn on expressions and declarations with an implied 'any' type.
        noResolve: true, // (optional) Skip resolution and preprocessing.
        removeComments: true, // (optional) Do not emit comments to output.
        concatenateOutput: false, // (optional) Concatenate and emit output to single file.
        // By default true if module option is omited, otherwise false.
      },

      // transforming the filenames
      transformPath: function(_path) {
        return _path.replace(/\.ts$/, '.js');
      },
    },

    scssPreprocessor: {
      options: {
        sourceMap: true,
      },
    },

    reporters: ['progress', 'coverage'],

    coverageReporter: {
      dir: 'reports/unit-tests/coverage',
      subdir: 'report',
      instrumenterOptions: {
        istanbul: { noCompact: true },
      },
      type: 'lcov',
      watermarks: {
        statements: [50, 90],
        branches: [1, 2],
        functions: [1, 2],
        lines: [1, 2],
      },
      check: {
        global: {
          statements: 60,
          lines: 65,
          functions: 60,
          branches: 45,
        },
      },
    },

sonar-project.properties

sonar.host.url=http://localhost:9000
sonar.projectKey=hello
sonar.projectName=hello
sonar.projectVersion=0.0.1

sonar.modules=hola

hola.sonar.projectBaseDir=hola/
hola.sonar.sources=src/
hola.sonar.exclusions=src/js/**/@types/*,src/js/**/**/resources/**/*
hola.sonar.tests=tests/
hola.sonar.language=ts
hola.sonar.typescript.lcov.reportPaths=reports/unit-tests/lcov.info
Agnaro commented 4 years ago

I am working on something very similar right now and seeing the same discrepancy. I believe my issue is caused by two things so you may be experiencing these too:

  1. My HTML and LCOV reports do not show results for any files that has no coverage (0%). SonarQube uses its own scan to files that should have coverage so it finds these files and shows the correct coverage which is lower than what the HTML report says.
  2. Because Sonar is using its own scan of files to include/exclude files, it may be including files you may want ignored. I believe there is a setting sonar.coverage.exclusions for this, but the exact name may depend on Sonar version.
AbdealiLoKo commented 4 months ago

Maybe this plugin can help: https://github.com/kopach/karma-sabarivka-reporter