search

thescientist13 / gallinago

Gallinago is a NodeJS package designed to help with the running and testing of CLIs.
MIT License
2 stars 1 forks source link

code coverage reports differ greatly between Windows / Linux and macOS #21

Open thescientist13 opened 3 years ago

thescientist13 commented 3 years ago

Type of Change

Summary

As part of landing Windows support in #20 , it was observed that nyc generated code coverage was different in Windows (lower) vs Ubuntu (higher).

Details

10 (or more!) point different sometimes. 😳

Windows

------------|---------|----------|---------|---------|--------------------
File        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s  
------------|---------|----------|---------|---------|--------------------
All files   |   85.45 |    76.67 |   84.62 |   85.45 |                    
 src        |     100 |      100 |     100 |     100 |                    
  index.js  |     100 |      100 |     100 |     100 |                    
 src/lib    |   84.91 |    76.67 |   84.62 |   84.91 |                    
  runner.js |   84.91 |    76.67 |   84.62 |   84.91 | 64-68,72-73,84,107 
------------|---------|----------|---------|---------|--------------------

=============================== Coverage summary ===============================
Statements   : 85.45% ( 47/55 )
Branches     : 76.67% ( 23/30 )
Functions    : 84.62% ( 11/13 )
Lines        : 85.45% ( 47/55 )
================================================================================

Ubuntu

------------|---------|----------|---------|---------|-------------------
File        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
------------|---------|----------|---------|---------|-------------------
All files   |   94.55 |    86.67 |     100 |   94.55 |                   
 src        |     100 |      100 |     100 |     100 |                   
  index.js  |     100 |      100 |     100 |     100 |                   
 src/lib    |   94.34 |    86.67 |     100 |   94.34 |                   
  runner.js |   94.34 |    86.67 |     100 |   94.34 | 66,82,107         
------------|---------|----------|---------|---------|-------------------

=============================== Coverage summary ===============================
Statements   : 94.55% ( 52/55 )
Branches     : 86.67% ( 26/30 )
Functions    : 100% ( 13/13 )
Lines        : 94.55% ( 52/55 )
================================================================================
Done in 6.88s.
thescientist13 commented 10 months ago

Double checking on this as part of #30 , and now it looks like MacOS is out of the loop now :/

windows-latetest

https://github.com/thescientist13/gallinago/actions/runs/7317090092/job/19932295698

------------|---------|----------|---------|---------|---------------------
File        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s   
------------|---------|----------|---------|---------|---------------------
All files   |   91.93 |     87.5 |     100 |   91.93 |                     
 src        |     100 |      100 |     100 |     100 |                     
  index.js  |     100 |      100 |     100 |     100 |                     
 src/lib    |   91.52 |    87.09 |     100 |   91.52 |                     
  runner.js |   91.52 |    87.09 |     100 |   91.52 | 61-63,[68](https://github.com/thescientist13/gallinago/actions/runs/7317090092/job/19932295698#step:6:69)-[72](https://github.com/thescientist13/gallinago/actions/runs/7317090092/job/19932295698#step:6:73),110-111 
------------|---------|----------|---------|---------|---------------------

=============================== Coverage summary ===============================
Statements   : 91.93% ( 114/124 )
Branches     : 87.5% ( 28/32 )
Functions    : 100% ( 5/5 )
Lines        : 91.93% ( 114/124 )
================================================================================

ubuntu-20.40

https://github.com/thescientist13/gallinago/actions/runs/7317090092/job/19932295636

------------|---------|----------|---------|---------|---------------------
File        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s   
------------|---------|----------|---------|---------|---------------------
All files   |   91.93 |     87.5 |     100 |   91.93 |                     
 src        |     100 |      100 |     100 |     100 |                     
  index.js  |     100 |      100 |     100 |     100 |                     
 src/lib    |   91.52 |    87.09 |     100 |   91.52 |                     
  runner.js |   91.52 |    87.09 |     100 |   91.52 | 61-63,[68](https://github.com/thescientist13/gallinago/actions/runs/7317090092/job/19932295636#step:6:69)-[72](https://github.com/thescientist13/gallinago/actions/runs/7317090092/job/19932295636#step:6:73),110-111 
------------|---------|----------|---------|---------|---------------------

=============================== Coverage summary ===============================
Statements   : 91.93% ( 114/124 )
Branches     : 87.5% ( 28/32 )
Functions    : 100% ( 5/5 )
Lines        : 91.93% ( 114/124 )
================================================================================

MacOS (local)

------------|---------|----------|---------|---------|---------------------------
File        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
------------|---------|----------|---------|---------|---------------------------
All files   |   90.32 |    84.84 |     100 |   90.32 |
 src        |     100 |      100 |     100 |     100 |
  index.js  |     100 |      100 |     100 |     100 |
 src/lib    |   89.83 |    84.37 |     100 |   89.83 |
  runner.js |   89.83 |    84.37 |     100 |   89.83 | 61-63,68-72,77-78,110-111
------------|---------|----------|---------|---------|---------------------------

=============================== Coverage summary ===============================
Statements   : 90.32% ( 112/124 )
Branches     : 84.84% ( 28/33 )
Functions    : 100% ( 5/5 )
Lines        : 90.32% ( 112/124 )
================================================================================
ERROR: Coverage for branches (84.84%) does not meet global threshold (85%)