Code Coverage doesn't fail with 0% cover.

n8ores commented 5 years ago

🐛 Bug Report

Running JEST with code coverage enabled does not fail if the code coverage is 0%

Image showing code cover running

To Reproduce

Enable JEST with the following configuration file

"jest": { "collectCoverage": true, "coverageDirectory": "../coverage", "testPathIgnorePatterns": [ "/node_modules/" ], "verbose": true, "coverageThreshold": { "global": { "branches": 50, "functions": 50, "lines": 50, "statements": 50 } } },

Then call your JEST script.

Expected behavior

JEST should fail with exit code 1 as the coverage limits are set to 50% and the test returned 0%.

thymikee commented 5 years ago

Please provide a repro somebody can download and triage.

n8ores commented 5 years ago

Dear Michal,

The bug template says providing a repro is optional (not required).

This can be reproduced using ANY vanilla install of NodeJS with the attacked package.json file provided.

Regards,

Nathan Parrott MBA, BEng Mechatronics

www.nathanparrott.com +61 408 263 865

On Thu, 13 Jun 2019 at 10:59, Michał Pierzchała notifications@github.com wrote:

Please provide a repro somebody can download and triage.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/facebook/jest/issues/8562?email_source=notifications&email_token=ABGTWCEXAL47RPDULAS6QVLP2J4FNA5CNFSM4HX37ZK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXULE4Q#issuecomment-501789298, or mute the thread https://github.com/notifications/unsubscribe-auth/ABGTWCHTI36BSCEIICSE4ILP2J4FNANCNFSM4HX37ZKQ .

thymikee commented 5 years ago

It's optional, that's why I'm not closing this issue :) (however I could do it, because you removed quite some important information, like which version of Jest you use, which Node, which OS). But it will make it easier for us to triage this.

For example, without a repro project, there's like 1% chances I'm gonna look at it. It's dramatically higher with a repro. Your choice! 🙂

n8ores commented 5 years ago

Fair enough:

Jest Version: 24.8.0 (latest) OS: Confirmed to fail on windows, Mac & Linux Node :10.15.3

stonefruit commented 5 years ago

Hello. I am also having an issue. Here is a repro: https://github.com/stonefruit/repro-jest-coverage.

npx create-react-app repro-jest-coverage

cd repro-jest-coverage

npm test -- --coverage

The above is used to create the repro. The coverage shown 'All files' with all 0s as described by @n8ores.

npm test -- --coverage

 PASS  src/App.test.js
  ✓ renders without crashing (22ms)

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.078s
Ran all test suites.

While the --watchAll flag shows the files.

npm test -- --coverage --watchAll

 PASS  src/App.test.js
  ✓ renders without crashing (23ms)

------------------|----------|----------|----------|----------|-------------------|
File              |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
------------------|----------|----------|----------|----------|-------------------|
All files         |     2.44 |        0 |     6.25 |     2.44 |                   |
 App.js           |      100 |      100 |      100 |      100 |                   |
 index.js         |        0 |      100 |      100 |        0 |              7,12 |
 serviceWorker.js |        0 |        0 |        0 |        0 |... 23,130,131,132 |
------------------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.562s
Ran all test suites.

Since there is a test for App.test.js, I think we should be expecting something that looks like (not accurate):

------------------|----------|----------|----------|----------|-------------------|
File              |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
------------------|----------|----------|----------|----------|-------------------|
All files         |     2.44 |        0 |     6.25 |     2.44 |                   |
 App.js           |      100 |      100 |      100 |      100 |                   |
------------------|----------|----------|----------|----------|-------------------|

kezarmader commented 4 years ago

Any update on this issue?

redeemefy commented 4 years ago

I'm having the same issue here.

"jest": "^24.9.0"
MBP 10.14.6
Nodejs 12.13.0

jest settings inside package.json

  "scripts": {
    "test": "jest",
    "test:watch": "npm run test -- --watch",
    "test:coverage": "npm run test -- --coverage --coverageDirectory=.jest",
    "generate:docs": "jsdoc -c ./jsdoc.conf.json"
  }

...

  "jest": {
    "rootDir": "./",
    "verbose": true,
    "collectCoverageFrom": [
      "**/*.{js}",
      "!**/node_modules/**",
      "!**/tools/**",
      "!**/build/**",
      "!**/docs/**"
    ],
    "coverageThreshold": {
      "global": {
        "statements": 100,
        "branches": 100,
        "functions": 100,
        "lines": 100
      }
    }
  },

outcome

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 13 passed, 13 total
Tests:       66 passed, 66 total
Snapshots:   0 total
Time:        1.525s
Ran all test suites.

redeemefy commented 4 years ago

I'm having the same issue here.

"jest": "^24.9.0"
MBP 10.14.6
Nodejs 12.13.0

jest settings inside package.json

  "scripts": {
    "test": "jest",
    "test:watch": "npm run test -- --watch",
    "test:coverage": "npm run test -- --coverage --coverageDirectory=.jest",
    "generate:docs": "jsdoc -c ./jsdoc.conf.json"
  }

...

  "jest": {
    "rootDir": "./",
    "verbose": true,
    "collectCoverageFrom": [
      "**/*.{js}",
      "!**/node_modules/**",
      "!**/tools/**",
      "!**/build/**",
      "!**/docs/**"
    ],
    "coverageThreshold": {
      "global": {
        "statements": 100,
        "branches": 100,
        "functions": 100,
        "lines": 100
      }
    }
  },

outcome

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 13 passed, 13 total
Tests:       66 passed, 66 total
Snapshots:   0 total
Time:        1.525s
Ran all test suites.

I found the solution to the problem. In the "jest": ... part within package.json substitute the ./ root for <rootDir>.

  "jest": {
    "collectCoverageFrom": [

      "<rootDir>/test/**/*.js"

    ],
    "coveragePathIgnorePatterns": [
      "<rootDir>/node_modules/",
      "<rootDir>/tools/",
      "<rootDir>/build/",
      "<rootDir>/docs/"
    ]

cansin commented 4 years ago

@diazgilberto that did not work for me unfortunately.

redeemefy commented 4 years ago

@diazgilberto that did not work for me unfortunately.

@cansin do you have your jest settings inside package.json?

cansin commented 4 years ago

Yes, I realized doing jest --coverage works, versus having collectCoverage: true inside jest.config.js does not.

redeemefy commented 4 years ago

Yes, I realized doing jest --coverage works, versus having collectCoverage: true inside jest.config.js does not.

Interesting is that even though I have all jest configuration inside package.json I still need an empty jest.config.js file in project root. If I remove that file, code coverage reports 0 again.

Now I'm having a different issue where coverage is reporting wrong % stating that I'm missing to test a particular line of code.

This is the code that is reporting wrong coverage

if (value.length === 0 && defaultValue !== undefined) {
  console.log('VALUE: ', value, ' DEFAULT: ', defaultValue)
  return defaultValue
}

console.log() inside the function that clearly shows that the function is getting tested.

  console.log lib/validation/validateString.js:38
    VALUE:    DEFAULT:  test

-------------------|----------|----------|----------|----------|-------------------|
File               |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
-------------------|----------|----------|----------|----------|-------------------|
All files          |      100 |    91.67 |      100 |      100 |                   |
 validateString.js |      100 |    91.67 |      100 |      100 |                37 |
-------------------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
Snapshots:   0 total
Time:        0.791s, estimated 2s
Ran all test suites matching /validateString/i.

npeham commented 4 years ago

Watch out for this warning:

ts-jest[versions] (WARN) Version 24.9.0 of jest installed has not been tested with ts-jest. If you're experiencing issues, consider using a supported version (>=22.0.0 <24.0.0). Please do not report issues in ts-jest if you are using unsupported versions.

Downgrading jest fixed the bug for me without changing the command. I also did not add the --watchAll flag.

ssrinath-sri commented 4 years ago

Hi, We can follow the below jest configuration...

"jest": { "moduleFileExtensions": [ "js", "json", "ts" ], "rootDir": ".", "testRegex": ".spec.ts$", "transform": { "^.+\.(t|j)s$": "ts-jest" }, "coverageDirectory": "./coverage", "testEnvironment": "node", "collectCoverage": true, "verbose": true }

bladechild commented 3 years ago

I'm having the same issue here.

"jest": "^24.9.0"
MBP 10.14.6
Nodejs 12.13.0

jest settings inside package.json

  "scripts": {
    "test": "jest",
    "test:watch": "npm run test -- --watch",
    "test:coverage": "npm run test -- --coverage --coverageDirectory=.jest",
    "generate:docs": "jsdoc -c ./jsdoc.conf.json"
  }

...

  "jest": {
    "rootDir": "./",
    "verbose": true,
    "collectCoverageFrom": [
      "**/*.{js}",
      "!**/node_modules/**",
      "!**/tools/**",
      "!**/build/**",
      "!**/docs/**"
    ],
    "coverageThreshold": {
      "global": {
        "statements": 100,
        "branches": 100,
        "functions": 100,
        "lines": 100
      }
    }
  },

outcome

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 13 passed, 13 total
Tests:       66 passed, 66 total
Snapshots:   0 total
Time:        1.525s
Ran all test suites.

I also had the similar issue before. My jest and ts-jest versions are: "ts-jest": "^26.3.0", "jest": "^26.4.2",

Here is my solution:

"collectCoverageFrom": [
  "**/*.{js}", -> "**/*.js"

this will show correct coverage.

Hope this helps.

bttger commented 3 years ago

I also had the 0% coverage problem when running jest with the --config= option. The problem was that jest assumes the configPath to be the root directory and thus doesn't find any files to check the coverage for.

My solution was simply to go higher in the directory tree:

{
  "rootDir": "../.",
  ...
}

brunolnetto commented 1 year ago

What is the final solution for this issue? My app provides a 0% coverage: https://github.com/web-needle/sappio

gladykov commented 1 year ago

Not sure if this is separate issue or same issue:

Steps:

Have a file ( http-error.ts ) not covered by tests on main branch
Change it and create new branch
Run tests

with parameter: --changedSince=main with config which forces scanning all files:

 collectCoverageFrom: [
        "<rootDir>/**/*.js",
        "<rootDir>/**/*.ts",
        "<rootDir>/**/*.cjs",
    ],

Result:

Tests are not run and changed file will not be listed and jest will not fail:

Output

No tests found related to files changed since "main".
Run Jest without `-o` or with `--all` to run all tests.
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |                   
----------|---------|----------|---------|---------|-------------------

Recovery step:

Change any other file covered by tests. This will trigger test run -> proper report, including files with 0 coverage:

--------------------------------|---------|----------|---------|---------|-------------------
File                            | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
--------------------------------|---------|----------|---------|---------|-------------------
All files                       |   45.45 |       50 |      50 |   45.45 |                   
 common                         |       0 |      100 |       0 |       0 |                   
  http-error.ts                 |       0 |      100 |       0 |       0 | 5-14              
 converter/html |     100 |       50 |     100 |     100 |                   
  include.ts               |     100 |       50 |     100 |     100 | 6                 
--------------------------------|---------|----------|---------|---------|-------------------

reno commented 1 year ago

🐛 Bug Report

Running JEST with code coverage enabled does not fail if the code coverage is 0%

To Reproduce

Enable JEST with the following configuration file

"jest": { "collectCoverage": true, "coverageDirectory": "../coverage", "testPathIgnorePatterns": [ "/node_modules/" ], "verbose": true, "coverageThreshold": { "global": { "branches": 50, "functions": 50, "lines": 50, "statements": 50 } } },

Then call your JEST script.

Expected behavior

JEST should fail with exit code 1 as the coverage limits are set to 50% and the test returned 0%.

Still having this problem on Jest 29.6.1

brunolnetto commented 1 year ago

@reno Suggestion: Try:

Removing tests and see if it is some syntax issue;
Using configuration of a functioning app (examples: sappio, prego)

cnwork commented 6 months ago

I'm having this same issue with:

Jest 29.7.0
Unbuntu 22.04.4

When running: jest --coverage --coverageThreshold='{"global":{"statements":"70"}}' --changedSince origin/master I get the same:

No tests found related to files changed since "origin/master".
Run Jest without `-o` or with `--all` to run all tests.
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |                   
----------|---------|----------|---------|---------|-------------------

In the commits that I made in my branch, I specifically modified .js files that don't have any tests. I should be able to somehow ask it to fail when 'No Tests are found'. Especially because I want this to fail in a GitHub Action to not let developers get a free pass on not writting any tests.

Thanks, C

L3V147H4N commented 2 months ago

I'm having the same issue on jest 29.7.0... the tests command pass regardless of test coverage.. no amount of tweaking and configuring fixed it... test always PASS if they succeed but even tho I'm specifying at least 80% coverage (in all the ways in can be configured) the commands never fails when under it.

Disregard my comment, the output was confusing, the test are PASSING but the command itself is indeed returning 1 when coverage is not met

jestjs / jest