Code coverage getting dropped when we combine code coverage from karma unit test case and from cypress integration test case

[rk011219]

Logs and screenshots I have angular 14 project where I am using karma framework for unit testing and cypress for integration test. I have separate code coverage for karma unit test and cypress integration test and I have combined both code coverage json files but code coverage got dropped after merging. I think code coverage is taking delta of them.

Karma Unit test case coverage:-

Cypress Integration code coverage:-

Combined Code coverage:-

If you see above reports, I am expecting combine report should show 100 % code coverage for components/sym-ui-sedr-actions because I have covered the component in integration test but it was showing delta. Is there any solution?

Versions

Cypress 10.8.0
"cypress-parallel": "0.9.1",
"@cypress/code-coverage": "3.10.0",
Angular 12.1.3
os : windows 10, linux
Node v12.22.3
npm v6.14.13
application instrumented using istambul and followed below lnk
https://lukas-klement.medium.com/implementing-code-coverage-with-angular-and-cypress-6ed08ed7e617

window.coverage : yes

Is there .nyc_output folder? Is there .nyc_output/out.json file. -yes Do you have any custom NYC settings in package.json (nyc object) or in other [NYC config files] yes

{
"extends": "@istanbuljs/nyc-config-typescript",
"all": true,
"excludeAfterRemap": true,
"reporter": ["html"],
"exclude": [
"./coverage/",
"cypress/",
"./dist/",
"/.spec.ts",
"/projects//test.ts",
"/projects//.js",
"./.js",
"./.config.ts",
"/projects//*.html",
"./src/**"
]
}

Do you run Cypress tests in a Docker container? NO

Describe the bug If you see above reports, I am expecting combine report should show 100 % code coverage for components/sym-ui-sedr-actions because I have covered the component in integration test but it was showing delta. Is there any solution?

FYI I am using Istanbul for instrumenting code for karma as well as for cypress

Example .nycrc.json

{
  "extends": "@istanbuljs/nyc-config-typescript",
  "all": true,
  "excludeAfterRemap": true,
  "exclude": [
    "./coverage/**",
    "cypress/**",
    "./dist/**",
    "**/*.spec.ts",
    "**/projects/**/test.ts",
    "**/projects/**/*.js",
    "./*.js",
    "./*.config.ts",
    "**/projects/**/*.html",
    "./src/**"
  ]
}

Krama.config.js

module.exports = function (config) {
  config.set({
    basePath: '',
    frameworks: ['jasmine', '@angular-devkit/build-angular'],
    plugins: [
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-firefox-launcher'),
      require('karma-jasmine-html-reporter'),
      require('karma-json-reporter'),
      require('karma-coverage'),
      require('karma-spec-reporter'),
      require('@angular-devkit/build-angular/plugins/karma'),
    ],
    client: {
      jasmine: {
        random: false,
        seed: '4321',
        oneFailurePerSpec: true,
        failFast: true,
        timeoutInterval: 30 * 1000,
      },
      clearContext: false, // leave Jasmine Spec Runner output visible in browser
    },
    jasmineHtmlReporter: {
      suppressAll: true, // removes the duplicated traces
    },
    coverageReporter: {
      check: {
        emitWarning: false,
        global: {
          statements: 85.42,
          branches: 70,
          functions: 84.37,
          lines: 81.76,
          excludes: ['src/**/*.js'],
        },
      },
      dir: require('path').join(__dirname, '../../../coverage/sym-ui/sedr'),
      instrumenterOptions: {
        istanbul: { noCompact: true },
      },
      reporters: [{ type: 'html' },{ type: 'json' }, { type: 'text-summary' }],
      subdir: '.',
      watermarks: {
        statements: [60, 85],
        functions: [60, 85],
        branches: [60, 85],
        lines: [60, 85],
      },
    },
    reporters: ['spec', 'progress', 'kjhtml', 'json'],
    port: 0,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: true,
    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    // Start these browsers, currently available:
    // - ChromeHeadlessNoSandbox
    // - Chrome
    // - ChromeCanary
    // - Firefox
    // - Opera
    // - Safari (only Mac)
    // - IE (only Windows)
    browsers: ['ChromeHeadlessNoSandbox'],
    customLaunchers: {
      ChromeHeadlessNoSandbox: {
        base: 'ChromeHeadless',
        flags: [
            '--disable-web-security',
            '--disable-gpu',
            '--no-sandbox'
        ]
      },
    },
    captureTimeout: 100 * 1000,
    reportSlowerThan: 5000,
    singleRun: false,
    restartOnFileChange: true,
    browserDisconnectTimeout: 100 * 1000,
    browserDisconnectTolerance: 8,
    browserNoActivityTimeout: 100 * 1000,
  });
};

[Scott-Fischer]

I'm seeing this as well with combined reports from Jest and Cypress

[maciek-ibm]

When running only Cypress the coverage i.e. in statements is 86.29% but after merging coverage from Cypress & Jest it drops to 80.17%. Does anyone here know how to fix this?

[MKhasib]

The issue is summarized with:

1. Unit tests (Jest) explicitly report uncovered lines.
2. E2E tests (Cypress) only report covered lines, omitting uncovered ones.
3. When merging coverage reports, this leads to underreporting of overall coverage coming from unit tests

The proper fix for this issue is for the e2e tests code-coverage plugin to report all lines the same way jest does

When I inspected the source instrumented code I found e2e test ignoring some lines from instrumenting causing the problem we are facing

[royooooooo]

The issue is summarized with:

1. Unit tests (Jest) explicitly report uncovered lines.
2. E2E tests (Cypress) only report covered lines, omitting uncovered ones.
3. When merging coverage reports, this leads to underreporting of overall coverage coming from unit tests

The proper fix for this issue is for the e2e tests code-coverage plugin to report all lines the same way jest does

When I inspected the source instrumented code I found e2e test ignoring some lines from instrumenting causing the problem we are facing

Hi @MKhasib ,

Thanks for sharing the idea for this, could you help sharing what documents you have been modified for this issue?

Thanks