Code Coverage DataCollector in .runsettings throws exception

atharkes commented 1 year ago

Description

I'm trying to make a custom .runsettings file but if I add a DataCollector tag with a friendlyName of Code Coverage vstest throws and exception and messes up the code coverage results.

Steps to reproduce

.runsettings file:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Exclude>
                <ModulePath>.*tests.dll$</ModulePath>
              </Exclude>
            </ModulePaths>
            <Sources>
              <Exclude>
                <Source>.*\\Migrations\\.*</Source>
              </Exclude>
            </Sources>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

Expected behavior

Settings to be correctly applied to the Code Coverage datacollector.

Actual behavior

Exception is thrown and code coverage results are incorrect/incomplete because of it.

Diagnostic logs

========== Starting test discovery ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22)
[xUnit.net 00:00:00.61]   Discovering: ***.Core.Tests
[xUnit.net 00:00:00.96]   Discovering: ***.Web.Tests
[xUnit.net 00:00:01.10]   Discovered:  ***.Core.Tests
[xUnit.net 00:00:01.11]   Discovering: ***.Infrastructure.EntityFramework.Tests
[xUnit.net 00:00:01.33]   Discovered:  ***.Web.Tests
[xUnit.net 00:00:01.47]   Discovered:  ***.Infrastructure.EntityFramework.Tests
========== Test discovery finished: 316 Tests found in 4,4 sec ==========
No C++ projects found in the solution. Dynamic C++ code coverage will be disabled. Static C++ code coverage will be enabled. If report is missing some C++ projects please enable /PROFILE linker option. More information: https://docs.microsoft.com/visualstudio/test/customizing-code-coverage-analysis#static-and-dynamic-native-instrumentation
========== Starting test run ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22)
[xUnit.net 00:00:00.27]   Starting:    ***.Core.Tests
[xUnit.net 00:00:05.66]   Finished:    ***.Core.Tests
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22)
[xUnit.net 00:00:00.29]   Starting:    ***.Web.Tests
[xUnit.net 00:00:01.99]   Finished:    ***.Web.Tests
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.22)
[xUnit.net 00:00:00.24]   Starting:    ***.Infrastructure.EntityFramework.Tests
[xUnit.net 00:00:01.51]   Finished:    ***.Infrastructure.EntityFramework.Tests
System.InvalidOperationException: Sequence contains more than one matching element
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.TestRunAttachmentsProcessing.TestRunAttachmentsProcessingManager.<ProcessAttachmentsAsync>d__9.MoveNext() in /_/src/Microsoft.TestPlatform.CrossPlatEngine/AttachmentsProcessing/TestRunAttachmentsProcessingManager.cs:line 164
System.InvalidOperationException: Sequence contains more than one matching element
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.TestRunAttachmentsProcessing.TestRunAttachmentsProcessingManager.<ProcessAttachmentsAsync>d__9.MoveNext() in /_/src/Microsoft.TestPlatform.CrossPlatEngine/AttachmentsProcessing/TestRunAttachmentsProcessingManager.cs:line 164
========== Test run finished: 316 Tests (316 Passed, 0 Failed, 0 Skipped) run in 5,7 sec ==========
========== Starting test discovery ==========
========== Test discovery skipped: All test containers are up to date ==========

Environment

Tested Visual Studio versions:

17.7.4
17.8.0 Preview 2.0

AB#1887046

atharkes commented 1 year ago

If I use automatic detection it also seems to detect the .runsettings file twice. Which would explain why there are two matches found where the invalid operation exception is thrown.

========== Starting test discovery ==========
========== Test discovery skipped: All test containers are up to date ==========
Using automatically detected runsettings file(s). To learn more visit https://aka.ms/vs-runsettings.
Using automatically detected runsettings file(s). To learn more visit https://aka.ms/vs-runsettings.
No C++ projects found in the solution. Dynamic C++ code coverage will be disabled. Static C++ code coverage will be enabled. If report is missing some C++ projects please enable /PROFILE linker option. More information: https://docs.microsoft.com/visualstudio/test/customizing-code-coverage-analysis#static-and-dynamic-native-instrumentation
========== Starting test run ==========
...

fhnaseer commented 1 year ago

@atharkes Adding uri should fix the issue. <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0">

atharkes commented 1 year ago

Thanks that seems to work <3

microsoft / codecoverage