Could not load type

[eddynaka]

Description

IMPORTANT: if the defect is reproduced only in a workflow from within the Visual Studio IDE then do not report the issue here - instead, please report it using Visual Studio's "Send Feedback" option that can be accessed from the Help menu OR using this link https://developercommunity.visualstudio.com.

For a defect reproducable from the vstest command line, describe the issue you've observed.

Today, I updated to the latest stable version (16.7.0) and, after that it isn't collecting code coverage anymore.

Steps to reproduce

What steps can reproduce the defect? Please share the setup, commandline for vstest.console, sample project, target framework etc.

run the command: dotnet test --collect:"Code Coverage" --results-directory:"TestResults" --configuration Release --no-build -- RunConfiguration.DisableAppDomain=true

Expected behavior

Share the expected output

Collect and don't show any error

Actual behavior

What is the behavior observed?

The following error is showing: Data collector 'Code Coverage' message: Failed to initialize code coverage datacollector with error: System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.Utilities.CodeCoverageRunSettingsProcessor' from assembly 'Microsoft.TestPlatform.Utilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

Diagnostic logs

Please share test platform diagnostics logs. Instructions to collect logs are here.
The logs may contain test assembly paths, kindly review and mask those before sharing.

Environment

Please share additional details about the test environment. Operating system, Build version of vstest.console

I'm running in GH Actions, windows-latest environment

[muiriswoulfe]

I'm seeing the same thing in our team's Azure DevOps build pipelines, which appears to be attributable to a set of NuGet package updates (although I can't ascertain which package update is responsible). If I revert the relatively large set of changes containing the NuGet package updates, the task works as before. Unfortunately, it is difficult to narrow down the precise update causing the error as many of the updates need to be made in lockstep due to package interdependencies.

The failure occurs in the Visual Studio Test task v2 when the "Code coverage enabled" option is selected.

Strangely, even after the changes are applied, some instances of the task continue to function with code coverage enabled. It appears that this issue only occurs when running the tests associated with binaries exhibiting certain characteristics (although I haven't been able to ascertain what exactly those characteristics are).

The error I'm seeing is as follows:

##[error]Data collector 'Code Coverage' message: Failed to initialize code coverage datacollector with error: System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.Utilities.CodeCoverageRunSettingsProcessor' from assembly 'Microsoft.TestPlatform.Utilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.Initialize(XmlElement configurationElement, IDataCollectionSink dataSink, IDataCollectionLogger logger)
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.OnInitialize(XmlElement configurationElement).
##[error]Data collector 'Code Coverage' message: Data collector 'Code Coverage' threw an exception during type loading, construction, or initialization: System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.Utilities.CodeCoverageRunSettingsProcessor' from assembly 'Microsoft.TestPlatform.Utilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.Initialize(XmlElement configurationElement, IDataCollectionSink dataSink, IDataCollectionLogger logger)
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.OnInitialize(XmlElement configurationElement)
##[error]   at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectorInformation.InitializeDataCollector()
##[error]   at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectionManager.LoadAndInitialize(DataCollectorSettings dataCollectorSettings, String settingsXml)..
##[error]Data collector 'Code Coverage' message: Failed to initialize code coverage datacollector with error: System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.Utilities.CodeCoverageRunSettingsProcessor' from assembly 'Microsoft.TestPlatform.Utilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.Initialize(XmlElement configurationElement, IDataCollectionSink dataSink, IDataCollectionLogger logger)
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.OnInitialize(XmlElement configurationElement).
##[error]Data collector 'Code Coverage' message: Data collector 'Code Coverage' threw an exception during type loading, construction, or initialization: System.TypeLoadException: Could not load type 'Microsoft.VisualStudio.TestPlatform.Utilities.CodeCoverageRunSettingsProcessor' from assembly 'Microsoft.TestPlatform.Utilities, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.Initialize(XmlElement configurationElement, IDataCollectionSink dataSink, IDataCollectionLogger logger)
##[error]   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.OnInitialize(XmlElement configurationElement)
##[error]   at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectorInformation.InitializeDataCollector()
##[error]   at Microsoft.VisualStudio.TestPlatform.Common.DataCollector.DataCollectionManager.LoadAndInitialize(DataCollectorSettings dataCollectorSettings, String settingsXml)..

[rouke-broersma]

Can confirm, we have the same issue. Pipeline here:

https://dev.azure.com/stryker-mutator/Stryker/_build/results?buildId=5131&view=logs&j=bf94e6da-eeef-50d1-100b-0ed7f4d964f2&t=6d3eec8c-0b41-5503-31a1-551142db5688&l=141

Started by this pull request where we update the following packages to 16.7.0

• Microsoft.TestPlatform.ObjectModel
• Microsoft.TestPlatform.TestHost
• Microsoft.TestPlatform
• Microsoft.TestPlatform.Portable
• Microsoft.TestPlatform.TranslationLayer
• Microsoft.NET.Test.Sdk
• Microsoft.CodeCoverage

[nohwnd]

Sorry for the inconvenience. @cvpoienaru is working on finding out what the issue is and seeing if there is any workaround.

[Phlow2001]

Just ran into this as well. Microsoft.NET.Test.Sdk 16.6.1 is fine.

[cvpoienaru]

@nohwnd @jakubch1 @eddynaka @muiriswoulfe @Mobrockers @Phlow2001 I was able to successfully replicate this issue on my machine and now I'm working on a fix for it.

I created a MSTest Test Project (.NET Core) project and used the latest Microsoft.NET.Test.Sdk NuGet package that has a dependency on the latest Microsoft.CodeCoverage NuGet. The issue replicates after running dotnet test --collect:"Code Coverage" --results-directory:"TestResults" --configuration Release --no-build -- RunConfiguration.DisableAppDomain=true using .NET SDK 3.1.300 but not using .NET SDK 3.1.400.

As a quick workaround, should it be possible for you, I recommend updating the .NET SDK version to at least 3.1.400. Let me know if you still see the issue after this please.

The reason this issue pops up in the first place is due to some change we did for Code Coverage (link here) that was causing the testhost process to crash under certain circumstances. On the way of fixing the issue in Microsoft.VisualStudio.TraceDataCollector.dll (part of Microsoft.CodeCoverage) we took a dependency on some functionality residing in Microsoft.TestPlatform.Utilities.dll (in this case part of .NET SDK) and as a consequence this incompatibility arose.

[rouke-broersma]

@cvpoienaru we cannot upgrade the dotnet core version as we build an open source dotnet cli tool so we have no control of where the tool is executed and which dotnet versions are available. But we have no need to upgrade right now so it's not a problem to wait on a fix.

[muiriswoulfe]

@cvpoienaru Thanks for the update.

Where is .NET SDK 3.1.400 available from? The latest release listed at https://dotnet.microsoft.com/download/dotnet-core/3.1 that I can see is 3.1.302. If it's a pre-release version, we're unfortunately not able to use that due to compliance reasons, although I expect we would not have to wait too long before an official release, in which case upgrading won't be a problem.

We're not under great pressure to get a fix here as we monitor code coverage through a variety of tools and so the data isn't entirely lost to us. But we would like to get it back via the build task eventually so that we can more quickly assess our project's code coverage status. Holding off until the official release of 3.1.400 is fine for us.

[Phlow2001]

We're on .NET Core 2.1

[eddynaka]

@muiriswoulfe , the new version got available! https://devblogs.microsoft.com/dotnet/net-core-august-2020/

[cvpoienaru]

@nohwnd @jakubch1 @eddynaka @muiriswoulfe @Mobrockers @Phlow2001 we have just released the latest version of TestPlatform, 16.8.0-preview-20200812-03, available on nuget.org.

This version contains a fix for this issue in the Microsoft.CodeCoverage NuGet, so provided you are able to use the latest set of NuGets you won't have to deal with the .NET SDK version bump requirement from the quick workaround above, instead you'll be able to use whatever .NET SDK version you were using until now.

[nohwnd]

@eddynaka could you (or anyone else) confirm the fix worked? We are considering releasing 16.7.1 and would like to avoid this passing our tests, but not yours 🙂

[eddynaka]

@nohwnd , sure! let me update my references and test in GH Actions. Will post the results soon.

[eddynaka]

@nohwnd , the new nuget worked fine. Don't if that's related. But, i have a second job that try to parse the .codecoverage file to .xml. And, with the new version it didn't work.

C:\Users\runneradmin\.nuget\packages\microsoft.codecoverage\16.8.0-preview-20200812-03\build\netstandard1.0\CodeCoverage\CodeCoverage.exe analyze /output:D:\a\opentelemetry-dotnet\opentelemetry-dotnet\TestResults\13dd0045-c9c6-4d62-8ecd-5f5e68f9d97c\runneradmin_fv-az171_2020-08-13.17_49_29.coverage.xml D:\a\opentelemetry-dotnet\opentelemetry-dotnet\TestResults\13dd0045-c9c6-4d62-8ecd-5f5e68f9d97c\runneradmin_fv-az171_2020-08-13.17_49_29.coverage

Below the entire pipeline: https://github.com/open-telemetry/opentelemetry-dotnet/pull/1074/checks?check_run_id=981636076

[nohwnd]

@cvpoienaru are you aware of change that could cause the above issue?

[cvpoienaru]

@nohwnd @eddynaka looking at the pipeline output for that particular job, I can see it fails because it's not able to find CodeCoverage.exe. Maybe the path you got is wrong ?

I tried running CodeCoverage.exe from the same NuGet on my machine and it worked.

Run .\build\process-codecoverage.ps1
  .\build\process-codecoverage.ps1
  shell: C:\windows\System32\WindowsPowerShell\v1.0\powershell.EXE -command ". '{0}'"
  env:
    OS: windows-latest
    DOTNET_ROOT: C:\hostedtoolcache\windows\dncs\3.1.401\x64
C:\Users\runneradmin
C:\Users\runneradmin\.nuget\packages\microsoft.codecoverage\16.8.0-preview-20200812-03\build\netstandard1.0\CodeCoverage\CodeCoverage.exe analyze /output:D:\a\opentelemetry-dotnet\opentelemetry-dotnet\TestResults\13dd0045-c9c6-4d62-8ecd-5f5e68f9d97c\runneradmin_fv-az171_2020-08-13.17_49_29.coverage.xml D:\a\opentelemetry-dotnet\opentelemetry-dotnet\TestResults\13dd0045-c9c6-4d62-8ecd-5f5e68f9d97c\runneradmin_fv-az171_2020-08-13.17_49_29.coverage
Invoke-Expression : The term 'C:\Users\runneradmin\.nuget\packages\microsoft.codecoverage\16.8.0-preview-20200812-03\bu
ild\netstandard1.0\CodeCoverage\CodeCoverage.exe' is not recognized as the name of a cmdlet, function, script file, or 
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try 
again.
At D:\a\opentelemetry-dotnet\opentelemetry-dotnet\build\process-codecoverage.ps1:7 char:5
+     Invoke-Expression $command
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\runner...odeCoverage.exe:String) [Invoke-Expression], CommandN 
   otFoundException
    + FullyQualifiedErrorId : CommandNotFoundException,Microsoft.PowerShell.Commands.InvokeExpressionCommand

##[error]Process completed with exit code 1.

[eddynaka]

hi @cvpoienaru , when I first looked at that i suspected of that.

But it's strange, since the only difference between this and the old version is the version number.

I will debug a little more and try to find out what is happening.

[nohwnd]

16.7.1 was released that addresses this. If you face this problem please update.

[nohwnd]

@abatishchev tagging you here, because you filed an issue on testfx for this. Please update to 16.7.1.

[eddynaka]

@nohwnd thank you! that version is working fine! already tested!

[nohwnd]

It was @cvpoienaru who ported the changes and did all the work, I am just a messenger trying to let everyone know 🙂

[eddynaka]

@cvpoienaru thank you! 👍