Could not find coverage.xml.zip when running Instrumented binaries

[nmoinvaz]

I am trying to figure out the easiest way to get the coverage data for our projects.

For my project I have multiple ways I need to generate test data. In the first case, it is through the standard dotnet test command. In the second case, I have to run the binaries themselves in order for the code paths to get hit. I've looked at a lot of the examples in the Wiki and they all use dotnet test. In my projects and solution I already included AltCover package.

I was thinking I would be able to simply run the executables that are generated in __Instrumented_MyProject.Console. However, when I try to do that it throws an error:

dotnet build MyProject.sln --configuration Release --property:AltCover=true --property:AltCoverReportFormat=opencover
cd MyProject.Console\bin\Debug\net8.0\__Instrumented_MyProject.Console
dotnet MyProject.Console.dll

Unhandled exception. System.IO.FileNotFoundException: Could not find file 'D:\MyProject.Console\coverage.xml.zip'.
File name: 'D:\MyProject.Console\coverage.xml.zip'
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at AltCover.Recorder.Counter.doFlushFile$cont@483(FSharpFunc`2 postProcess, FSharpFunc`2 pointProcess, Boolean own, Dictionary`2 counts, ReportFormat format, String report, FSharpOption`1 output, Unit unitVar) in D:\a\altcover\altcover\AltCover.Recorder\Base.fs:line 483
   at AltCover.Recorder.Counter.doFlushFile(FSharpFunc`2 postProcess, FSharpFunc`2 pointProcess, Boolean own, Dictionary`2 counts, ReportFormat format, String report, FSharpOption`1 output) in D:\a\altcover\altcover\AltCover.Recorder\Base.fs:line 470
   at AltCover.Recorder.Instance.I.flushAll@249.Invoke(Boolean own) in D:\a\altcover\altcover\AltCover.Recorder\Recorder.fs:line 250
   at AltCover.Recorder.Instance.I.withMutex[a](FSharpFunc`2 f) in D:\a\altcover\altcover\AltCover.Recorder\Recorder.fs:line 218
   at AltCover.Recorder.Instance.I.arg10@245(Dictionary`2 counts, Unit unitVar0) in D:\a\altcover\altcover\AltCover.Recorder\Recorder.fs:line 249
   at AltCover.Recorder.Instance.I.flushAll[a](a _arg1) in D:\a\altcover\altcover\AltCover.Recorder\Recorder.fs:line 245
   at AltCover.Recorder.Instance.I.flushCounter[a](Close finish, a _arg1) in D:\a\altcover\altcover\AltCover.Recorder\Recorder.fs:line 461
   at AltCover.Recorder.Instance.I.doExit@474.Invoke(Object _arg1, EventArgs a) in D:\a\altcover\altcover\AltCover.Recorder\Recorder.fs:line 474

Thanks again for your help in advance.

[SteveGilham]

I'm not sure what it is you are actually doing, as dotnet build [solution] takes no notice of the altcover properties; there is no __Instrumented folder after such an operation.

I suspect that what you want to do is use altcover as a command-line tool, as discussed in e.g. the modes of operation wiki page. For this style of operation you can give the path to the altcover binaries in the nuget cache, but it is probably more convenient to either explicitly nuget it to a local folder; or better, use the altcover global tool.

[SteveGilham]

With a simple solution like this one doing

dotnet test TestProject1.sln --configuration Release --property:AltCover=true --property:AltCoverReportFormat=opencover

gives an error trying to test the console application but going to the instrumented folder and running the console application gives and the coverage.xml file in the console app project directory contains the obvious visit

        <Class>
          <Summary numSequencePoints="1" visitedSequencePoints="0" numBranchPoints="1" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="1" minCyclomaticComplexity="1" visitedClasses="0" numClasses="1" visitedMethods="0" numMethods="1" minCrapScore="2" maxCrapScore="2" />
          <FullName>&lt;StartupCode$ConsoleApp1&gt;.$Program</FullName>
          <Methods>
            <Method visited="false" cyclomaticComplexity="1" nPathComplexity="0" sequenceCoverage="0" branchCoverage="0" isConstructor="false" isStatic="true" isGetter="false" isSetter="false" crapScore="2">
              <Summary numSequencePoints="1" visitedSequencePoints="0" numBranchPoints="1" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="1" minCyclomaticComplexity="1" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="1" minCrapScore="2" maxCrapScore="2" />
              <MetadataToken>100663298</MetadataToken>
              <Name>System.Void &lt;StartupCode$ConsoleApp1&gt;.$Program::main@()</Name>
              <FileRef uid="36" />
              <SequencePoints>
                <SequencePoint vc="1" uspid="0" ordinal="0" offset="0" sl="2" sc="1" el="2" ec="24" bec="0" bev="0" fileid="36" />
              </SequencePoints>
              <BranchPoints />
              <MethodPoint xsi:type="SequencePoint" vc="0" uspid="0" ordinal="0" offset="0" sl="2" sc="1" el="2" ec="24" bec="0" bev="0" fileid="36" />
            </Method>
          </Methods>
        </Class>

being the results of a run in "classic" mode (just the visit counts filled in, not the summaries); with no filtering, report file also includes the FSharp.Core assembly.

So I'm not sure exactly what it is you were doing; but knowing that it would have to be more than what was specified above, which doesn't even ask for the coverage file to be zipped.

The fix issue #209 will mean that the console program will not be tested, or even instrumented stand-alone; however by going to the instrumented folder under the test program, it is possible to run the instrumented console app, with the results, in this case, going in classic mode to the test project's coverage file.

[nmoinvaz]

So I'm not sure exactly what it is you were doing;

What I was doing was:

• Adding altcover package to all projects
• Building in Debug mode (this creates the __Instrumented folder)
• Trying to run the executables from the __Instrumented folder.

Now, after further research about the Modes of Operation, I realize:

• I don't have to add altcover package, but instead can use altcover global tool.
• Build in Release mode
• Instrument the binaries into a separate folder before running them using global tool.

I think it might have helped me a bit if the Modes of Operation wiki page was clearer to understand somehow but I'm not sure how. But it could also help if there was simple examples of how to use altcover in GitHub Actions or in CI environment.

I've gotten it all running now. Thank you for your help and for your work on this great project!

Please feel free to close this issue if you see that I what I was doing initially is not a supported mode/method of running altcover.

[SteveGilham]

Thanks for the comments on the documentation. It's partly a result of the original AltCover (2010-2017) having been a plain .Net Framework application; with the dotnet test integration being a later extension which tries to join in the automatic behaviour of the newer tool, which has in the subsequent years become the default way of doing things rather than something new and exotic.

This is the part that I'm not sure about, and would like to understand before I close the issue

Building in Debug mode (this creates the __Instrumented folder)

because just building shouldn't create that folder.

That requires running the VSTest target, within which the Prepare task that creates that directory is executed, and dotnet build doesn't go near the testing target. Secondly, to be looking for a zipped coverage file, it would need $(AltCoverZipFile) to have been set to "true", which isn't the sort of thing that happens by accident.

[nmoinvaz]

This is the part that I'm not sure about, and would like to understand before I close the issue

Building in Debug mode (this creates the __Instrumented folder)

I tried to reproduce it but was not able to. I must have been mistaken how that folder was created. Sorry.