istanbuljs / nyc

the Istanbul command line interface
https://istanbul.js.org/
ISC License
5.54k stars 353 forks source link

nyc test coverage on branches is not accurate - Seems to be happening with nyc@v15, mocha@10.2.0 and Node@v18.17 #1540

Open varunrajasekhar opened 6 months ago

varunrajasekhar commented 6 months ago

Link to bug demonstration repository

Expected Behavior

Branch Coverage to be giving more accurate value

Observed Behavior

Branch Coverage seems to be wrong from our end

Troubleshooting steps

The Nyc branch coverage seems to be a bit off to me. I think it is giving me wrong branch coverages.The value and the tests dont seem to be valid to me. We recently upgrade our legacy app from istanbul to nyc. Istanbul had much more valid coverage listed.

Our app is using Node 18.17.0 (cannot downgrade - only option is to upgrade) Nyc - 15.1.0, corresponding mocha => 10.2.0, sinon => 1.17.3 (Since we use sinon stubs)

Our "npm run test" scripts is "test": "nyc node_modules/.bin/mocha -- -u bdd -R spec test/server//.js --exit",

nyc.config.js { "all": true, "include": ["test/server/controllers/.js","test/server/services/.js","test/server/utils/**.js"], "exclude":["test/server/index","test/server/coverage/lcov-report", "target", "server"], "reporter":["html", "cobertura", "text", "text-summary"], "check-coverage": true, "report-dir":"./target/coverage" }

What am I supposed to do on this to have the correct branch coverage ?

I am also attaching the istanbul test report

This one is Istanbul Coverage report

Environment Information

Screenshot 2023-12-21 at 10 17 02 AM

This one is Nyc report

Screenshot 2023-12-21 at 10 15 26 AM This one is istanbul report

paste the output here

varunrajasekhar commented 6 months ago

@jamestalmage @novemberborn @rmg

varunrajasekhar commented 6 months ago

Seems to me that branch coverage is not getting updated in the nyc

image

Please check this here. The Object is always empty @Lalem001 @XhmikosR @addaleax @coreyfarrell @bcoe

jjuannn commented 6 months ago

I think i'm having a similar problem.- Seems that the report branch coverage is not being so accurate since it marks me like i'm not going through certaing lines that i'm already covering.

image

image

Did you managed to make it work of any way? @varunrajasekhar

varunyarlagaddaTR commented 6 months ago

@jjuannn - I think it is the way nyc intercepts the branch coverage etc. I see in the core logic of nyc methods, the branchCoverage Object is unable to retrieve the correct mapping (always shows empty obj for all the files). I couldnt figure out why it does so. May it has something to do with istanbul v/s nyc configurations. Still waiting for response from the @supporters, @maintainers

zhangboli51 commented 5 months ago

having exactly same issue node21 + nyc 15 +mocha 10

zhangboli51 commented 5 months ago

My issue is resolved , in my case it's caused by @babel/node. which is used to support ES6 modules, but nyc seems confused with some ES6 module. So I have to stick with CommonJS modules, and this not only impact file where it contains ES6 module , but also other files not using ES6.... (so weird) for example i have ES6 code :

export default helper

then I have to change to

module.exports = helper;