search

gotwarlost / istanbul

Yet another JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests. Supports all JS coverage use cases including unit tests, server side functional tests and browser tests. Built for scale.
Other
8.7k stars 786 forks source link

Huge difference between istanbul 1.0-alpha2 and isparta #616

Closed tleunen closed 6 years ago

tleunen commented 8 years ago

I'm currently testing istanbul v1.0 alpha2 instead of isparta for a project but I'm getting a huge difference in the report... Same codebase, just different tool:

isparta:

Statements   : 81.06% ( 5548/6844 ), 662 ignored
Branches     : 75.01% ( 2467/3289 ), 800 ignored
Functions    : 72.87% ( 1050/1441 ), 117 ignored
Lines        : 60.44% ( 1948/3223 )

istanbul:

Statements   : 56.22% ( 1817/3232 )
Branches     : 41.98% ( 599/1427 )
Functions    : 47.96% ( 364/759 )
Lines        : 56.9% ( 1772/3114 )
jamesplease commented 8 years ago

@tleunen it appears as if code coverage for transpiled code is all over the place right now. No two tools produce the same result 😛

This thread is a good read: https://github.com/jmcriffey/babel-istanbul/issues/56

The last comment suggests that this project is accurate, but it's pretty new and might produce accurate results in all cases.

If anyone figures out the best solution for coverage, do let me know. I'm trying to figure out what to use for the Babel boilerplate (https://github.com/babel/generator-babel-boilerplate/issues/330)

gotwarlost commented 8 years ago

Hi @jmeas - I'm currently collaborating with @dtinth on an instrumentation library that uses his ideas in the __coverage__ plugin and preserves the behavior of istanbul's current instrumenter.

That work is going on here and it should be ready "soon".

https://github.com/istanbuljs/istanbul-lib-instrument/issues/4 https://github.com/istanbuljs/istanbul-lib-instrument/pull/5