Issue with DLL loading when using runsettings

[marcopierobon]

I'm trying to specify a runsettings file from VS to add some coverage rules.

Since I added it, one of the tests started to fail.

The test uses reflection to load a dependent assembly and enforce some validation on its classes.

String FluentMigratorFullName = "MyAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null";
var fluentMigratorAssemblyName = new AssemblyName(FluentMigratorFullName);
var fluentMigratorAssembly = Assembly.Load(fluentMigratorAssemblyName);

The error I get is:

System.IO.FileNotFoundException: Could not load file or assembly 'MyAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

This works fine locally.

I know there is a AssemblyResolution parameter, but the location would be different at each execution as it depends on the agent picking up the build.

This is my .runsettings

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Configurations that affect the Test Framework -->
  <RunConfiguration>

  </RunConfiguration>

  <!-- Configurations for data collectors -->
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Exclude>
                <ModulePath>.*MyModule.*</ModulePath>
              </Exclude>
            </ModulePaths>

            <!-- We recommend you do not change the following values: -->
            <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>

          </CodeCoverage>
        </Configuration>
      </DataCollector>

    </DataCollectors>
  </DataCollectionRunSettings>

  <!-- Parameters used by tests at runtime -->
  <TestRunParameters>

  </TestRunParameters>

  <!-- Adapter Specific sections -->

  <!-- MSTest adapter -->
  <MSTest>
    <MapInconclusiveToFailed>false</MapInconclusiveToFailed>
    <CaptureTraceOutput>true</CaptureTraceOutput>
  </MSTest>

</RunSettings>

UPDATE:

I noticed that the "Run tests in parallel on multi-core machines" was active before.

Turning it back on fixed the issue. Any idea why?

Also, turning this on with the runsetting makes the test report unavailable to the build, hence the coverage metric cannot be enforced.

[ReneSchumacher]

Hi,

I'm not sure why the option to run tests in parallel should influence assembly resolution. Neither do I understand, why this option would prevent the build from accessing the test results or coverage report. I will check a couple things tomorrow and talk to the testing tools team if necessary.

In the meantime you could try to activate the fusion log on your build agent (assuming you can run your build on a private agent). You can find more information at https://docs.microsoft.com/en-us/dotnet/framework/tools/fuslogvw-exe-assembly-binding-log-viewer. With fusion log enabled you can run your build once with parallel tests and once without and you should see the different assembly resolution behavior in the logs. This might point to the root cause.