Incorrect Jest coverage

[stephane-arista]

When using @swc/jest to transpile ts to commonjs and running jest --coverage certain branches are shown as not covered (console logging in these branches show that tests do run the code path). Using babel to transpile and run the tests shows the correct coverage.

[pspeter3]

I added this to my Jest configuration in package.json which seemed to help.

"jest": {
    "collectCoverage": true,
    "transform": {
      "^.+\\.(t|j)sx?$": [
        "@swc/jest",
        {
          "sourceMaps": true
        }
      ]
    }
  }

[krutoo]

@pspeter3 Thank you for your code snippet, it helps, but now there is another problem:

It looks like not covered code highlights are not displayed correctly.

There is an example of coverage report from project with:

• jest@27.3.1
• @swc/core@1.2.118
• @swc/jest@0.2.11

the yellow highlight here says that the comma is supposedly not covered

[sebald]

@krutoo ran into the same issues and guessed it is caused by the conversion to older JS syntax. We could fix these false-positives by targeting a newer ES version.

  transform: {
    '^.+\\.(t|j)sx?$': [
      '@swc/jest',
      {
        jsc: {
          target: 'es2021',
        },
        sourceMaps: true,
      },
    ],
  },

[krutoo]

@sebald Thank you, looks like it realy works with target: "es2021"

[JoshuaKGoldberg]

Another reproduction here: https://github.com/typescript-eslint/tslint-to-eslint-config/pull/1367

I ended up only needing target: "es2021". Thanks!

[kdy1]

I think swc reusing span for various places may cause this.

Can you try https://sokra.github.io/source-map-visualization/#custom by manually invoking swc?

[bobaaaaa]

@kdy1 does this help? Maybe the Object.keys(_node).forEach produces the mismatch? Jest complains that this line is not covered: export * as node from './BodyNodesBuilder';

Edit: tested with

• @swc/jest 0.2.20
• jest 27.5.1

[kdy1]

@bobaaaaa Can you share some code? I think the test file would be enough, as it does not have good sourcemap.

[kdy1]

Oh... Maybe assumption about monotonic increment of source map position can be the cause. I'll add it to my tasklist

[bobaaaaa]

@kdy1 You can find it here: https://gist.github.com/bobaaaaa/3649b3a7e6312793a257bf67c500128a Let me know if something is missing.

(thx for investigating ❤️)

[kdy1]

You need sourceMaps: true or sourceMaps: "inline". I verified that it's working

[bobaaaaa]

@kdy1 Hm, I tested both sourceMaps in .swcrc and as an option in the jest.config.json. Both did not work for me. Even with target: "es2021".

[kdy1]

Hmm... Source maps are valid, maybe emitted tokens without sourcemap entry can be the cause I guess? Some sourcemap libraries have bugs related to it.

[bobaaaaa]

@kdy1 I updated the gist with the generated .js + .js.map files: https://gist.github.com/bobaaaaa/3649b3a7e6312793a257bf67c500128a

[kdy1]

I'm not sure why does jest can't understand the sourcemap. I need to dig into jest to know the source map library it uses...

[kdy1]

Can you try the latest version of @swc/core? (v1.2.155)

Patches in https://github.com/swc-project/swc/pull/4007 are very likely to fix this issue.

[bobaaaaa]

@kdy1 I tested the new version. Unfortunately, its still not fixed :(

Tested with es2020 & es2021 + sourceMaps: true & sourceMaps: "inline". I updated all files in my gist build with v1.2.155: https://gist.github.com/bobaaaaa/3649b3a7e6312793a257bf67c500128a

(Keep in mind, this is not a huge blocker/issue for us. Still thx for looking into this)

[kdy1]

Can you try the latest version? (v1.2.156) There was a bug fix for a module that only contains export * from './foo' and such module is super common, so I think it can be the cause of this.

[wight554]

Can confirm same issue with vitest + swc with all recommended configs and latest version (v1.2.156)

[bobaaaaa]

@kdy1 problem is still there with v1.2.156

[bobaaaaa]

@kdy1 I saw you created another fix (https://github.com/swc-project/swc/pull/4074) 👍 Because in 1.2.157 the issue is still present.

[kdy1]

I changed the milestone, thanks!

[kdy1]

I triggered publishing of the new version

[sscode02]

I see you fixed this in v1.2.158, but the problem persists after I update, is it my configuration problem?

transform: {
    '^.+\\.(t|j)sx?$': [
      '@swc/jest',
      {
        jsc: {
          target: 'es2021',
        },
        sourceMaps: true,
      },
    ],
  },

Edit: tested with

• @nestjs 7.6.15
• @swc/jest 0.2.20
• @swc/core 1.2.158
• jest 26.6.3

[wight554]

In my case all decorator occurrences in code are marked as uncovered branches Only way to get proper coverage report is to use tsc for test compilation

[wight554]

My reproduction repo for reference: https://github.com/wight554/blog-template/tree/swc-test Coverage report is included in coverage folder (see controllers)

---------------------------------------|---------|----------|---------|---------|-------------------
File                                   | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
---------------------------------------|---------|----------|---------|---------|-------------------
All files                              |   99.61 |    67.21 |     100 |   99.61 |                   
 server/auth                           |     100 |      100 |     100 |     100 |                   
  AuthController.ts                    |     100 |      100 |     100 |     100 |                   
  AuthService.ts                       |     100 |      100 |     100 |     100 |                   
 server/auth/guards                    |      80 |      100 |     100 |      80 |                   
  JwtAuthGuard.ts                      |      80 |      100 |     100 |      80 | 1                 
  LocalAuthGuard.ts                    |      80 |      100 |     100 |      80 | 1                 
 server/comment                        |     100 |     37.5 |     100 |     100 |                   
  CommentController.ts                 |     100 |     37.5 |     100 |     100 | 19,25-26,33       
  CommentService.ts                    |     100 |      100 |     100 |     100 |                   
 server/comment/dto                    |     100 |      100 |     100 |     100 |                   
  CreateCommentDto.ts                  |     100 |      100 |     100 |     100 |                   
  UpdateCommentDto.ts                  |     100 |      100 |     100 |     100 |                   
 server/constants                      |     100 |      100 |     100 |     100 |                   
  controllers.ts                       |     100 |      100 |     100 |     100 |                   
 server/crypto                         |     100 |      100 |     100 |     100 |                   
  CryptoService.ts                     |     100 |      100 |     100 |     100 |                   
 server/decorators                     |     100 |      100 |     100 |     100 |                   
  UserDecorator.ts                     |     100 |      100 |     100 |     100 |                   
 server/enums                          |     100 |      100 |     100 |     100 |                   
  MongoError.ts                        |     100 |      100 |     100 |     100 |                   
 server/interceptors                   |   97.56 |      100 |     100 |   97.56 |                   
  ...ooseClassSerializerInterceptor.ts |   97.56 |      100 |     100 |   97.56 | 1                 
 server/post                           |     100 |    41.17 |     100 |     100 |                   
  PostController.ts                    |     100 |    41.17 |     100 |     100 | ...58-59,66,73-76 
  PostService.ts                       |     100 |      100 |     100 |     100 |                   
 server/post/dto                       |     100 |      100 |     100 |     100 |                   
  CreatePostDto.ts                     |     100 |      100 |     100 |     100 |                   
  UpdatePostDto.ts                     |     100 |      100 |     100 |     100 |                   
 server/user                           |     100 |       50 |     100 |     100 |                   
  UserController.ts                    |     100 |       50 |     100 |     100 | 28,31,37-40       
  UserService.ts                       |     100 |      100 |     100 |     100 |                   
 server/user/dto                       |     100 |      100 |     100 |     100 |                   
  CreateUserDto.ts                     |     100 |      100 |     100 |     100 |                   
  UpdateUserDto.ts                     |     100 |      100 |     100 |     100 |                   
 src                                   |     100 |      100 |     100 |     100 |                   
  app.tsx                              |     100 |      100 |     100 |     100 |                   
  logo.tsx                             |     100 |      100 |     100 |     100 |                   
---------------------------------------|---------|----------|---------|---------|-------------------

tsc reports 100% for all

[bobaaaaa]

yes, can confirm. This is still not fixed. But /* istanbul ignore next */ now works :)

[wight554]

still not fixed in @swc/core 1.2.159 if it was meant to be fixed

[chortis]

yes, can confirm. This is still not fixed. But /* istanbul ignore next */ now works :)

@bobaaaaa

Do you find that while this works, it's not quite as well as babel-jest? For example I cannot ignore a whole file or large function blocks that have nested functions.

I'm able to ignore these files in collectCoverageFrom so not a blocker, but something I have observed.

[krutoo]

for no reason in one of the projects it is reproduced again

example:

dependencies:

{
  "@swc/core": "^1.2.207",
  "@swc/jest": "^0.2.21",
  "jest": "^28.1.1"
}

[kdy1]

I think it's related to recent changes of module passes.

[krutoo]

@kdy1 Hmm, with @swc/core@1.2.198 it works fine

[kdy1]

I'm working on this but this seems like a quite tricky issue

[kdy1]

I improved sourcemap a bit with https://github.com/swc-project/swc/pull/5569

Current status:

Still not sure how coverage tools count execution counts, though

[markerikson]

Implementation-wise?

By rewriting the code so that this:

// line 1
const q = 42;
// line 2
const x = q + 5;
// line 3
const z = x * 2;

becomes:

globalLinesCounter[1]++;
const q = 42;
globalLinesCounter[2]++
const x = q + 5;
globalLinesCounter[3]++;
const z = x * 2;

ie, literally incrementing per-line counters in a global object

[bobaaaaa]

@kdy1 I tested the new version. Unfortunately, its still not fixed :(

Tested with es2020 & es2021 + sourceMaps: true & sourceMaps: "inline". I updated all files in my gist build with v1.2.155: https://gist.github.com/bobaaaaa/3649b3a7e6312793a257bf67c500128a

(Keep in mind, this is not a huge blocker/issue for us. Still thx for looking into this)

Hey @kdy1 my reported issue here is now fixed for me with "@swc/jest": "0.2.22"

[jsardev]

@kdy1 Unfortunately I still get wrong (false-positive) coverage reports on 1.3.4 😢

nyc/istanbul

c8

.swcrc

{
  "$schema": "https://json.schemastore.org/swcrc",
  "jsc": {
    "parser": {
      "syntax": "typescript",
      "decorators": true
    },
    "transform": {
      "decoratorMetadata": true
    }
  },
  "sourceMaps": true
}

.tsconfig

{
  "extends": "@tsconfig/node16",
  "compilerOptions": {
    "baseUrl": ".",
    "resolveJsonModule": true,
    "outDir": "build",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "sourceMap": true,
    "paths": {
      "@/*": ["src/*"],
      "@modules/*": ["src/modules/*"],
      "@test/*": ["test/*"]
    }
  }
}

[jsardev]

@kdy1 Can I help you somehow with this? I mean, I probably won't fix it as I have no knowledge about how the process works, but maybe some kind of minimal reproduction repo would help? 🤔

[kdy1]

Yes, it will definitely help. Ideally if I can invoke jest to get coverage for single file, I can test/ensure that jest reports 100% coverage

[jsardev]

@kdy1 Here it is: https://github.com/sarneeh/swc-project-swc-issues-3854

When I was creating it I noticed that the false-positives go away when I turn off decoratorMetadata option (which is unfortunately required when using dependency injection frameworks like typedi). Hopefully that will help you somehow 🙏

[kdy1]

The clue about decoratorMetadata really helps! I already fixed codegen and it means some transform is dropping source map information, and now I know which pass is doing so. Thank you!

[stefee]

I've noticed that jest coverage will report code branches inside helper functions as being uncovered lines unless the externalHelpers:true options is specified.

          jsc: {
            externalHelpers: true,
          },

Adding this option improved my test coverage significantly since the helper code is no longer included in the coverage scan. I'm not sure if anyone else has encountered this?

I opened a PR to add this option in @swc-node/jest but it probably needs further investigation/discussion https://github.com/swc-project/swc-node/pull/673

[tkeiyama]

I had the same issue. But when I changed sourceMaps: true to sourceMaps: "inline", jest started to collect coverage correctly.

My config is:

// jest.config.js
const config = {
  transform: {
    "^.+\\.(t|j)sx?$": ["@swc/jest", { ...swcConfig, sourceMaps: "inline" }],
  },
}

Ofc, Adding the option to .swcrc works as well.

Hope this helps people.

[stefee]

I’m actually using @swc-node/jest which already sets sourcemaps to inline by default.

https://github.com/swc-project/swc-node/blob/c2b8389ae6b325392f9320d2ecefd2b08b95d23b/packages/core/index.ts#L33

[960590968]

I had the same issue. But when I changed sourceMaps: true to sourceMaps: "inline", jest started to collect coverage correctly.

My config is:

// jest.config.js
const config = {
  transform: {
    "^.+\\.(t|j)sx?$": ["@swc/jest", { ...swcConfig, sourceMaps: "inline" }],
  },
}

Ofc, Adding the option to .swcrc works as well.

Hope this helps people.

I had same issue. Could you show swcConfig at here? @tkeiyama

[tkeiyama]

I’m actually using @swc-node/jest which already sets sourcemaps to inline by default.

This is true... :|

I had same issue. Could you show swcConfig at here?

Sure. @960590968 Here are my config files for testing. And I'm using React. Basically, swcConfig is configs from .swcrc

jest.config.js

```js const { readFileSync } = require("fs"); const swcConfig = JSON.parse(readFileSync(`${__dirname}/.swcrc`, "utf-8")); /** @type {import('jest').Config} */ const config = { rootDir: ".", collectCoverage: true, collectCoverageFrom: [ "src/**/*.{ts,tsx}", "!src/**/*.stories.tsx", ], coverageThreshold: { global: { statements: 90, branches: 90, functions: 90, lines: 90, }, }, setupFilesAfterEnv: ["/jest.setup.js"], transform: { "^.+\\.(t|j)sx?$": ["@swc/jest", { ...swcConfig }], }, testEnvironment: "jsdom", testMatch: ["/src/**/*.test.{ts,tsx}"], }; module.exports = config; ```

jest.setup.js

```js import "@testing-library/jest-dom"; ```

.swcrc

```json { "sourceMaps": "inline", "minify": true, "jsc": { "parser": { "syntax": "typescript", "jsx": true }, "transform": { "react": { "runtime": "automatic" } } } } ```

Dependencies

// package.json ```json { ... "dependencies": { "clsx": "^1.2.1", "react": "^18.2.0", "react-dom": "^18.2.0" }, "devDependencies": { "@storybook/addon-essentials": "^6.5.13", "@storybook/addon-postcss": "^2.0.0", "@storybook/builder-vite": "^0.2.5", "@storybook/react": "^6.5.13", "@swc/core": "^1.3.20", "@swc/jest": "^0.2.23", "@testing-library/dom": "^8.19.0", "@testing-library/jest-dom": "^5.16.5", "@testing-library/react": "^13.4.0", "@testing-library/react-hooks": "^8.0.1", "@testing-library/user-event": "^14.4.3", "@types/jest": "^29.2.3", "@types/node": "^18.11.9", "@types/react": "^18.0.25", "@types/react-dom": "^18.0.9", "autoprefixer": "^10.4.13", "dprint": "^0.33.0", "husky": "^8.0.0", "jest": "^29.3.1", "jest-environment-jsdom": "^29.3.1", "lint-staged": "^13.0.4", "npm-run-all": "^4.1.5", "postcss": "^8.4.19", "storybook-addon-themes": "^6.1.0", "tailwindcss": "^3.2.4", "typescript": "^4.9.3" } } ```

[klutzer]

I've tested with "sourceMaps": "inline" but didn't work.

.swcrc:

{
  "sourceMaps": "inline",
  "jsc": {
    "target": "es2021",
    "parser": {
      "syntax": "typescript",
      "decorators": true,
      "dynamicImport": true
    },
    "keepClassNames": true,
    "transform": {
      "legacyDecorator": true,
      "decoratorMetadata": true
    }
  }
}

package.json:

"devDependencies": {
    ...
    "@swc/core": "^1.3.20",
    "@swc/jest": "^0.2.23"
},
"jest": {
    "moduleFileExtensions": [
      "js",
      "json",
      "ts"
    ],
    "rootDir": ".",
    "testRegex": ".*\\.spec\\.ts$",
    "transform": {
      "^.+\\.(t|j)s$": "@swc/jest"
    },
    "collectCoverageFrom": [
      "src/**/*.{ts,js}"
    ],
    "moduleNameMapper": {
      "^@app/(.*)$": "<rootDir>/src/$1",
      "^@test/(.*)$": "<rootDir>/test/$1"
    },
    "coverageDirectory": "./coverage",
    "coverageReporters": [
      "json-summary",
      "text",
      "lcov"
    ],
    "coverageThreshold": {
      "global": {
        "branches": 100,
        "functions": 100,
        "lines": 100,
        "statements": 100
      }
    },
    "testEnvironment": "node"
  },
  "engines": {
    "node": "^16.16.0"
  }
}

[danr-za]

same as @klutzer here, same jest configuration. Might be related to decorators as previously

[danr-za]

btw, not sure its 100% related, but in regards to ignore comments, this is the generated code I see:

function myFn(type) {
    -   /* istanbul ignore next */
    -   cov_shygh21on().f[5]++;
    -   cov_shygh21on().s[14]++;
    -   return `${type}Tag`;
    - }

[Krzywy14]

Useing coverageProvider: 'v8' will probably solve the problem. Solution funded at stackoverflow https://stackoverflow.com/a/74851858/8770040