oven-sh / bun

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one
https://bun.sh
Other
74.29k stars 2.77k forks source link

Bun test code coverage - Not returning exit code 1 when under threshold #7367

Open dominicporteous opened 11 months ago

dominicporteous commented 11 months ago

What version of Bun is running?

1.0.13+f5bf67bd1

What platform is your computer?

Darwin 21.6.0 x86_64 i386

What steps can reproduce the bug?

bunfig.toml

[test]
coverage = true
coverageThreshold = { line = 0.9, function = 0.9 }
$ bun test 

....
--------------------------------------------------------|---------|---------|-------------------
File                                                    | % Funcs | % Lines | Uncovered Line #s
--------------------------------------------------------|---------|---------|-------------------
All files                                               |   55.20 |   69.58 |
...
--------------------------------------------------------|---------|---------|-------------------

 138 pass
 0 fail
 486 expect() calls

$ echo $?
0

What is the expected behavior?

bun test returns exit code 1

What do you see instead?

bun test returns exit code 0

Additional information

Also tried with CI=true and seen same behaviour

dhythm commented 10 months ago

I'm facing the similar issue at v1.0.21. coverageThreshold = 0.98 works well, but coverageThreshold = { line = 0.98, function = 0.98 } doesn't work.

----------------------------|---------|---------|-------------------
File                        | % Funcs | % Lines | Uncovered Line #s
----------------------------|---------|---------|-------------------
All files                   |   95.85 |   96.20 |
 ...
----------------------------|---------|---------|-------------------
 120 pass
 0 fail
 618 expect() calls
Ran 120 tests across 26 files. [530.00ms]
Warning: run-commands command "npx --yes bun test --tsconfig-override ./tsconfig.spec.json" exited with non-zero status code

 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————
----------------------------|---------|---------|-------------------
File                        | % Funcs | % Lines | Uncovered Line #s
----------------------------|---------|---------|-------------------
All files                   |   95.85 |   96.20 |
 ...
----------------------------|---------|---------|-------------------
 120 pass
 0 fail
 618 expect() calls
Ran 120 tests across 26 files. [468.00ms]

 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————
leizaf commented 9 months ago

btw not a bug, see #8864

jakeboone02 commented 5 months ago

Looks like this issue was fixed at some point:

image