search

coverlet-coverage / coverlet

Cross platform code coverage for .NET
MIT License
2.99k stars 386 forks source link

"The process cannot access the file, because it is being used by another process" error during attempt to collect coverage #857

Closed AntiPasha closed 4 years ago

AntiPasha commented 4 years ago

The error message for MSBuild intergration 

C:\Users\xxx\.nuget\packages\coverlet.msbuild\2.8.1\build\coverlet.msbuild.targets(36,5): error : One or more errors occurred. (The process cannot access the file 'D:\Work\xxx\yyy\test\yyy.Tests\bin\Debug\netcoreapp2.1\zzz.Dto.dll' because it is being used by another process.)[D:\Work\xxx\yyy\test\yyy.Tests\yyy.Tests.csproj]
C:\Users\xxx\.nuget\packages\coverlet.msbuild\2.8.1\build\coverlet.msbuild.targets(36,5): error :    at Coverlet.Core.Helpers.RetryHelper.Do[T](Func`1 action, Func`1 backoffStrategy, Int32 maxAttemptCount) in c:\git\coverletToRelease\src\coverlet.core\Helpers\RetryHelper.cs:line 59 [D:\Work\xxx\yyy\test\yyy.Tests\yyy.Tests.csproj]
C:\Users\xxx\.nuget\packages\coverlet.msbuild\2.8.1\build\coverlet.msbuild.targets(36,5): error :    at Coverlet.Core.Helpers.RetryHelper.Retry(Action action, Func`1 backoffStrategy, Int32 maxAttemptCount) in c:\git\coverletToRelease\src\coverlet.core\Helpers\RetryHelper.cs:line 28 [D:\Work\xxx\yyy\test\yyy.Tests\yyy.Tests.csproj]
C:\Users\xxx\.nuget\packages\coverlet.msbuild\2.8.1\build\coverlet.msbuild.targets(36,5): error :    at Coverlet.Core.Helpers.InstrumentationHelper.RestoreOriginalModule(String module, String identifier) in c:\git\coverletToRelease\src\coverlet.core\Helpers\InstrumentationHelper.cs:line 219 [D:\Work\xxx\yyy\test\yyy.Tests\yyy.Tests.csproj]
C:\Users\xxx\.nuget\packages\coverlet.msbuild\2.8.1\build\coverlet.msbuild.targets(36,5): error :    at Coverlet.Core.Coverage.GetCoverageResult() in c:\git\coverletToRelease\src\coverlet.core\Coverage.cs:line 219 [D:\Work\xxx\yyy\test\yyy.Tests\yyy.Tests.csproj]
C:\Users\xxx\.nuget\packages\coverlet.msbuild\2.8.1\build\coverlet.msbuild.targets(36,5): error :    at Coverlet.MSbuild.Tasks.CoverageResultTask.Execute() in c:\git\coverletToRelease\src\coverlet.msbuild.tasks\CoverageResultTask.cs:line 113 [D:\Work\xxx\yyy\test\yyy.Tests\yyy.Tests.csproj]

The same behavior for VSTest integration 

---> (Inner Exception #1) System.IO.IOException: The process cannot access the file 'D:\Work\xxx\yyy\test\yyy.Tests\bin\Debug\netcoreapp2.1\zzz.dll' because it is being used by another process.
   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
   at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite)
   at Coverlet.Core.Helpers.FileSystem.Copy(String sourceFileName, String destFileName, Boolean overwrite) in c:\git\coverletToRelease\src\coverlet.core\Helpers\FileSystem.cs:line 33
   at Coverlet.Core.Helpers.InstrumentationHelper.<>c__DisplayClass10_0.<RestoreOriginalModule>b__0() in c:\git\coverletToRelease\src\coverlet.core\Helpers\InstrumentationHelper.cs:line 215
   at Coverlet.Core.Helpers.RetryHelper.<>c__DisplayClass0_0.<Retry>b__0() in c:\git\coverletToRelease\src\coverlet.core\Helpers\RetryHelper.cs:line 26
   at Coverlet.Core.Helpers.RetryHelper.Do[T](Func`1 action, Func`1 backoffStrategy, Int32 maxAttemptCount) in c:\git\coverletToRelease\src\coverlet.core\Helpers\RetryHelper.cs:line 52<---

It happens all the time, when I try to collect coverage for xunit test projects (with NUnit everything works fine suddenly) using default commands, like dotnet test --collect:"XPlat Code Coverage" or dotnet test /p:CollectCoverage=true

Adding -nr:false -m:1 arguments to MSBuild didn't help, as well as running commands with separate csproj file. I've used the latest versions of coverlet, xunit and Microsoft.NET.Test.Sdk packages

<PackageReference Include="coverlet.collector" Version="1.2.1">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="coverlet.msbuild" Version="2.8.1">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.6.1" />
<PackageReference Include="Moq" Version="4.14.1" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.1">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>

Installed SDKs are below

.NET Core SDK (reflecting any global.json):
 Version:   3.1.202
 Commit:    6ea70c8dca

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.202\

Host (useful for support):
  Version: 3.1.4
  Commit:  0c2e69caa6

.NET Core SDKs installed:
  2.1.806 [C:\Program Files\dotnet\sdk]
  3.1.202 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Migrated from comments of 725 issue

AntiPasha commented 4 years ago

I've collected some events using Process Monitor, there no processes, except dotnet.exe and testhost.exe, accessing the file. During usual test run (without coverage) there is only file open, query information and locking file with only readers by testhost.exe. When coverage is run, there are a file reading, writing and locking file with only readers by both processes. After file is locked by testhost.exe, dotnet.exe is returning a sharing violation result when attempt to use a file for generic read and write. It seems that both processes have something like a race condition, when file is locked for reading by one process and second process attempts to access it with read/write access.

Attached two files with events, first one for run with /p:CollectCoverage=true, second one with regular dotnet test. Attempt to run with --collect:"XPlat Code Coverage" shows the same behavior with msbuild integration coverlet_msbuild.xlsx dotnet.test.xlsx

MarcoRossignoli commented 4 years ago

Can you try only with collectors and enable verbose logging and attach logs(before check if there are sensitive data inside) https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Troubleshooting.md#collectors-integration? I want to understand if vstest plat logging show some contention exception.

AntiPasha commented 4 years ago

Hi, please find verbose log for collectors attached. Thanks for your patience log.host.20-05-29_01-00-26_57408_4.txt log.txt log.datacollector.20-05-29_01-00-25_75553_4.txt

MarcoRossignoli commented 4 years ago

Thanks

MarcoRossignoli commented 4 years ago

I cannot repro and without stack trace of thread that blocks files it's hard to understand the reason, I've got a temporary idea to try, we could avoid to raise error in case of issue during file restoring, so at least you can get a coverage file output. Could work for the moment?The only drawback is that the files inside bin/debug will remain instrumented and so no more usable for further test run.

MarcoRossignoli commented 4 years ago

That errors happens at the end of all coverage when we copy back original(non instrumented) files, but at that time coverage data are correctly gathered.

MarcoRossignoli commented 4 years ago

@AntiPasha I'm working on a workaround/solution, can you try with this package(collectors driver dotnet test --collect:"XPlat Code Coverage")

nuget feed source https://f.feedz.io/marcorossignoli/coverletunofficial/nuget/index.json package version 1.3.1-preview.6.g1f7fe02d13

This is the feed I use for preview/investigation.

Branch https://github.com/coverlet-coverage/coverlet/compare/master...MarcoRossignoli:issue857

AntiPasha commented 4 years ago

@MarcoRossignoli Thanks a lot for the update, I've tried your package and it seems to work fine, I didn't seen any exceptions, report is generated in random folder under TestResults, but in cobertura format. I thought that json is default one, because I didn't use any settings

MarcoRossignoli commented 4 years ago

For collector the default one is cobertura as explained in the guide

MarcoRossignoli commented 4 years ago

If you confirm that amend works well I can merge and you can use nightly build until next official release

AntiPasha commented 4 years ago

Does it work only for VSTest integration? What about msbuild one?

MarcoRossignoli commented 4 years ago

Works for both the engine in shared, but give me some minutes...I'll upload also msbuild version so you can try and confirm.

MarcoRossignoli commented 4 years ago

Uploaded coverlet.msbuild.2.9.1-preview.6.g1f7fe02d13 same feeds.

BTW my advice is to use collectors because has got better integration and doesn't suffer of some issues https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#1-vstest-stops-process-execution-earlydotnet-test

Please let me know if works as expected!

AntiPasha commented 4 years ago

@MarcoRossignoli MSBuild integration works as well, thanks in advance!

MarcoRossignoli commented 4 years ago

Great will PR asap!

cruikshj commented 4 years ago

I do not think this has been released yet. Is there a release with this fix coming soon?

MarcoRossignoli commented 4 years ago

We should be near new release, we're discussing new versioning schema and after we'll release.

cc: @tonerdo

xJom commented 3 years ago

Good thread! I think we are on the same problem here. I managed to install the coverlet.collector.1.3.1-preview.6.g1f7fe02d13, but I can't seem to find the coverlet.msbuild.2.9.1-preview.6.g1f7fe02d13. Did you update to another preview?

xJom commented 3 years ago

Maybe I need the coverlet.msbuild, because I have a single test in a single class, causing this lockup, every time, it is not even intermittent for me. Currently, it is not open source code. I will see if I can produce an open example from the code.

The output is: ---> (Inner Exception #11) System.IO.IOException: The process cannot access the file 'C:\Jenkins\workspace\SharedUI\SharedUI-develop_builder\UnitTests\XUnit.Coverlet.Collector\bin\Debug\netcoreapp3.1\MessageFormatInterface.dll' because it is being used by another process. at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite) at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite) at Coverlet.Core.Helpers.FileSystem.Copy(String sourceFileName, String destFileName, Boolean overwrite) in c:\git\coverlet\src\coverlet.core\Helpers\FileSystem.cs:line 33 at Coverlet.Core.Helpers.InstrumentationHelper.<>c__DisplayClass12_0.<RestoreOriginalModule>b__0() in c:\git\coverlet\src\coverlet.core\Helpers\InstrumentationHelper.cs:line 218 at Coverlet.Core.Helpers.RetryHelper.<>c__DisplayClass0_0.<Retry>b__0() in c:\git\coverlet\src\coverlet.core\Helpers\RetryHelper.cs:line 26 at Coverlet.Core.Helpers.RetryHelper.Do[T](Func1 action, Func`1 backoffStrategy, Int32 maxAttemptCount) in c:\git\coverlet\src\coverlet.core\Helpers\RetryHelper.cs:line 52<---

--- End of inner exception stack trace --- at Coverlet.Collector.DataCollection.CoverageManager.InstrumentModules() in c:\git\coverlet\src\coverlet.collector\DataCollection\CoverageManager.cs:line 70 at Coverlet.Collector.DataCollection.CoverletCoverageCollector.OnSessionStart(Object sender, SessionStartEventArgs sessionStartEventArgs) in c:\git\coverlet\src\coverlet.collector\DataCollection\CoverletCoverageCollector.cs:line 140.

Test Run Successful. Total tests: 1 Passed: 1 Total time: 3,6722 Seconds `

MarcoRossignoli commented 3 years ago

@xJom I apologize but we're in the middle of nightly migration and at the moment you can try with my unofficial feed I use to debug https://f.feedz.io/marcorossignoli/coverletunofficial/nuget/index.json

Keep the eyes on https://github.com/coverlet-coverage/coverlet/issues/969

xJom commented 3 years ago

Thanks! I managed to run it with 3.0.0-preview.9 for both coverlet.msbuild and coverlet.collector, but still one file is locking up. I have run it with both Process Monitor and search for locks in the ProcessExplorer - but I find nothing.

I also run with all parameters:

dotnet test /nodeReuse:false -m:1 .\XUnit.Coverlet.Collector.csproj /p:ParallelizeTestCollections=false /p:CopyLocalLockFileAssemblies=true /p:ExcludeFromCodeCoverage=MessageFormatInterface --collect:"XPlat Code Coverage" But still no luck.

I also tried to turn off the Real-time protection of Microsoft Defender, but no difference. Since this is repeatable, it might have something to do with the structure of the test, but I am out of clues right now. Will discuss with a colleague.

MarcoRossignoli commented 3 years ago

Strange...keep me informed on it, also pls in case open a new issue this is boiled.

aling93 commented 3 years ago

I am also running into this and the preview package has fixed it. Any plans to have a stable release for this? Thanks!

danjcosg commented 3 years ago

^^ Still running into this too

MarcoRossignoli commented 3 years ago

@danjcosg can you open a new issue?

jerotas2005 commented 2 years ago

This is a year later, and this problem still exists. Newest Coverlet Nuget, .NET 6 (or .NET Core 3.1 same). Was it ever fixed? I'm not allowed to use unofficial Nuget feeds so I haven't tried that.

MarcoRossignoli commented 2 years ago

The problem will always exist if some tool open handle to files when we want to restore the dll at the end, unfortunately we're not in control of that in any way.

One solution can be skip the restore(with some flag opt-in) if we're sure that the test build will be discarded at the end of the test and not used anymore outside testing.