Closed 39547361 closed 5 years ago
Looks like the coverage results aren't being properly flushed to disk on process termination; the same underlying issue as #42
release/v4.0.659 (that fixed #42) does not fix this issue
Actually, issue #42 was re-opened a couple of weeks before this issue was raised. Getting a repro on that is next on my list when I get a chance to resume work on this project.
Sorry for misleading comment. What i meant to say was: This issue still present with an older altcover release/v4.0.659 it might be relevant in so far that issue #42 is not occurring with altcover release/v4.0.659
As the only change near the relevant code between 4.0.659 and 5.0.663 was a change in the static linkage of the F# core assembly, the original "fix" may have been more a "working by coincidence".
Given that the whole system relies on any outstanding data being flushed during the process shutdown handler, I can see a possibility where very small or very large programs-under-test work, but moderately sized ones fail with zero coverage, so I need to test that hypothesis.
I've not been able to get any reliable repro, but I have done some tuning on the recording side in release 5.0.665, which may ameliorate this problem.
Thank you so far for your effort. Unfortunately 5.0.665 does not fix this particular issue. I created a private sample project and sent you an invite in case you are interested.
I was afraid that I wouldn't have fixed it; I have a very intermittent case which showed up for the first time in a while after I'd made the release.
I'm not seeing your invite anywhere obvious from my side, though.
The invite appeared a mailbox sub-folder, rather than anywhere on my GitHub profile. I hadn't been expecting that.
Thank you for that. It reproduces on Windows.
The first thing I noticed in looking at the coverage.xml file was that every assembly was marked skippedDueTo="Filter"
; the second was that the module under test, AltcoverTestPCL
wasn't even present there.
I proceeded to investigate by the following steps
AltCover.exe --opencover --linecover --assemblyExcludeFilter="^((?!AltcoverTestPCL).)*$" --outputDirectory=__Instrumented
using a regex to simply exclude every assembly not containing AltcoverTestPCL
in its namex86
and x64
folders to __Instrumented
(a limitation; I could have used --inplace
in step 4 instead); other platforms will just place libe_sqlite3.so
in the Debug
folder where it will get copied as expectedAltCover.exe Runner --recorderDirectory __Instrumented --executable ../../../packages/NUnit.ConsoleRunner.3.9.0/tools/nunit3-console.exe --./__Instrumented/NUnit.AltcoverTest.dll
to point NUnit at the instrumented test system in __Instrumented
.11,931 visits recorded in 00:00:00.2904547 (41,077 visits/sec)
A total of 11,931 visits recorded
Coverage statistics flushing took 0.38 seconds
Visited Classes 43 of 43 (100)
Visited Methods 83 of 84 (98.81)
Visited Points 543 of 582 (93.3)
Visited Branches 0 of 84 (0)
==== Alternative Results (includes all methods including those without corresponding source) ====
Alternative Visited Classes 43 of 54 (79.63)
Alternative Visited Methods 83 of 132 (62.88)
Thank you very much for resolving my misconfiguration. Have a great day sir!
FWIW, the issue in point 5 of my comment of the 8th Feb 2019 is belatedly resolved -- the manual copy of the x86 and x64 folders is no longer needed, as this is done automatically since today's release of v8.2.825; at the time I was too busy focussing on the zero coverage issue to heed my own words about this being a limitation.
For some strange reason i get 0% coverage with altocover (monocov reports 90%). Am i missing something? Any hints would be greatly appreciated.
altcover.5.0.663 MacOSX 17.7.0.0 CLR Version: 4.0.30319.42000
Step 1: create pcl and nunit test dll
Step 2
Step 3
Output:
Generated files:
coverage xml looks fine
Step 4:
Output: