search

JuliaCI / Coverage.jl

Take Julia code coverage and memory allocation results, do useful things with them
MIT License
174 stars 68 forks source link

Coverage.jl reporting lines surrounded by covered code as not covered #261

Closed vlandau closed 4 years ago

vlandau commented 4 years ago

I'm noticing issues with a coverage report for my Julia package submitted to codecov showing lines of code as not covered that are surrounded by covered code. There is no control flow in this section of code -- basically, if lines n and n+2 were covered, then line n+1 must have also been covered, but the report is stating otherwise.

It could be something on codecov's end also, but wanted to post this here to make devs aware. Here is the coverage report with an example highlighted for reference.

baggepinnen commented 4 years ago

Was the line counted at all? Maybe that line was elided by the compiler in case it counts as neither a hit or a miss. That usually applieds to constant definitions etc.

DilumAluthge commented 4 years ago

@vlandau Can you try running your test suite with inlining disabled, and see if that fixes the problem? I opened a pull request here: https://github.com/Circuitscape/Omniscape.jl/pull/14

vlandau commented 4 years ago

Thanks @DilumAluthge! Trying it now.

vlandau commented 4 years ago

It does seem that the coverage errors were significantly worse when the after_script in travis was run after the documentation stage. I added some control flow to the after_script to only have it run after the Test stage on Linux, and the problem was reduced significantly. There is still one case, here.

Note that this is all before trying with no inlining. I'll post the results here once that finishes running.

vlandau commented 4 years ago

The previous comment above seems to still apply. There are two cases with skipped lines: here, and here. I can live with this just fine for now, but a fix probably does exist.

DilumAluthge commented 4 years ago

In the second case (line 168 of run_omniscape.jl), are you sure that the line is being run?

vtjnash commented 4 years ago

Expected to be fixed by https://github.com/JuliaLang/julia/pull/34254

vlandau commented 4 years ago

In the second case (line 168 of run_omniscape.jl), are you sure that the line is being run?

:man_facepalming: I thought I had a test that covered that line but I actually did not! Whoops.

vlandau commented 4 years ago

Appears to have been fixed (presumably by JuliaLang/julia#34254). Line count is still not perfect (some lines are skipped), but there are no longer any false negatives in the report (e.g. lines that were covered that are being reported as not covered). Closing this since it was about those false negatives.