search

coverlet-coverage / coverlet

Cross platform code coverage for .NET
MIT License
2.99k stars 386 forks source link

Running coverlet global tool over dotnet build intermittently fails to open file when calculating coverage #491

Closed jkoritzinsky closed 5 years ago

jkoritzinsky commented 5 years ago

In my project https://github.com/SharpGenTools/SharpGenTools I've recently switched to using Coverlet for my code-coverage instrumentation. I've been encountering an intermittent issue where the Coverlet global tool is unable to open the hits file since it is being used by another process.

This only happens when I'm using the Coverlet global tool to instrument my MSBuild tasks while running dotnet build.

Here's the repro steps that I have:

  1. Clone SharpGenTools
  2. In Powershell run ./build ; ./test. That will build SharpGenTools and then run the tests. It will likely fail to build the outerloop tests because the process will be unable to access the hits file.

Alternatively, run the following commands from the cloned build:

./build
./build/deploy-test-packages Debug
./build/build-outerloop-native
dotnet restore ./SdkTests/SdkTests.sln
coverlet ".\SdkTests\RestoredPackages\sharpgentools.sdk\2.0.0-local\tools\netstandard1.3\SharpGenTools.Sdk.dll" -t "dotnet" -a "build ./SdkTests/SdkTests.sln /nodeReuse:false --no-restore" -f opencover -o ./artifacts/coverage/outerloop.xml --include-test-assembly --include-directory .\SdkTests\RestoredPackages\sharpgentools.sdk\2.0.0-local\tools\netstandard1.3

The last command will fail with the issue.

I can also privately send you a minidump reproing the issue if you want.

jkoritzinsky commented 5 years ago

Sometimes you'll get a different crash (with the same exception) when the instrumentation tracker in MSBuild loses the race and can't access the hits file to save the data on the shutdown of the dotnet.exe process. I also have a minidump saved for this scenario.

cbroxton commented 5 years ago

Could you post some more info about the error you're getting? I think we're getting an error which sounds similar to what you described in our Azure Devops pipeline, see below...

2019-07-23T12:53:55.0463811Z Test Run Successful. 2019-07-23T12:53:55.0481876Z Test execution time: 30.7144 Seconds 2019-07-23T12:53:55.1195193Z 2019-07-23T12:53:55.1377050Z Calculating coverage result... 2019-07-23T12:53:55.5143800Z C:\Users\DevOpsAgent2\.nuget\packages\coverlet.msbuild\2.6.3\build\coverlet.msbuild.targets(41,5): error : Unable to read beyond the end of the stream. [C:\agent_2\_work\1\s\xxx\xxx\xxx\xxx\xxx.Test.csproj] 2019-07-23T12:53:55.5144045Z C:\Users\DevOpsAgent2\.nuget\packages\coverlet.msbuild\2.6.3\build\coverlet.msbuild.targets(41,5): error : at System.IO.BinaryReader.FillBuffer(Int32 numBytes) [C:\agent_2\_work\1\s\xxx\xxx\xxx\xxx\xxx.Test.csproj] 2019-07-23T12:53:55.5144131Z C:\Users\DevOpsAgent2\.nuget\packages\coverlet.msbuild\2.6.3\build\coverlet.msbuild.targets(41,5): error : at System.IO.BinaryReader.ReadInt32() [C:\agent_2\_work\1\s\xxx\xxx\xxx\xxx\xxx.Test.csproj] 2019-07-23T12:53:55.5154396Z C:\Users\DevOpsAgent2\.nuget\packages\coverlet.msbuild\2.6.3\build\coverlet.msbuild.targets(41,5): error : at Coverlet.Core.Coverage.CalculateCoverage() in C:\Users\toni\Workspace\coverlet\src\coverlet.core\Coverage.cs:line 253 [C:\agent_2\_work\1\s\xxx\xxx\xxx\xxx\xxx.Test.csproj] 2019-07-23T12:53:55.5154672Z C:\Users\DevOpsAgent2\.nuget\packages\coverlet.msbuild\2.6.3\build\coverlet.msbuild.targets(41,5): error : at Coverlet.Core.Coverage.GetCoverageResult() in C:\Users\toni\Workspace\coverlet\src\coverlet.core\Coverage.cs:line 129 [C:\agent_2\_work\1\s\xxx\xxx\xxx\xxx\xxx.Test.csproj] 2019-07-23T12:53:55.5154809Z C:\Users\DevOpsAgent2\.nuget\packages\coverlet.msbuild\2.6.3\build\coverlet.msbuild.targets(41,5): error : at Coverlet.MSbuild.Tasks.CoverageResultTask.Execute() in C:\Users\toni\Workspace\coverlet\src\coverlet.msbuild.tasks\CoverageResultTask.cs:line 84 [C:\agent_2\_work\1\s\xxx\xxx\xxx\xxx\xxx.Test.csproj] 2019-07-23T12:53:55.7264131Z ##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1

MarcoRossignoli commented 5 years ago

I don't know if they are same problem, we have a know issue for `Unable to read... "https://github.com/tonerdo/coverlet/issues/210 that should be fixed by collectors, I'm OOF at the moment I'll take a look next week, sorry for the delay guys and thanks for help on investigation

jkoritzinsky commented 5 years ago

The issue that I found seemed to be caused by the fact that dotnet build defaults to parallel builds. The following two things happened:

I worked around it for me by running the build process with -m:1 to force the build to run sequentially on one node.

cbroxton commented 5 years ago

Thanks for that, I've added the -m:1 flag to the dotnet build step of our Azure Devops pipeline and based on initial tests it seems to have solved the issue. I couldn't find any docs on how exactly that flag works though.

StevenTCramer commented 5 years ago

@cbroxton I'm hitting the same issue. Can you explain what you mean but adding the -m:1 flag?

Adding it where? I am looking at the docs and it doesn't seem to exist.

MarcoRossignoli commented 5 years ago

-m:1 forces sequential builds using only one core

some infos https://docs.microsoft.com/en-us/visualstudio/msbuild/building-multiple-projects-in-parallel-with-msbuild?view=vs-2019

StevenTCramer commented 5 years ago

The process cannot access the file 'C:\agent_work\6\s\test\TestApp.EndToEnd.Tests\bin\Debug\netcoreapp3.0\BlazorState.dll' because it is being used by another process.

Still happens with the maxcpucount set to 1

If anyone cares to review.... https://timewarpenterprises.visualstudio.com/Blazor-State/_build/results?buildId=593 without coverlet they all pass.

MarcoRossignoli commented 5 years ago

@StevenTCramer any news? Close this issue as solved, I mean with a workaround on coverlet issue with parallel instrumentation/test run This will be likely resolved if we complete https://github.com/tonerdo/coverlet/issues/357