[BUG]: DotNetCoreCLI@2 to run tests occasionally fails with error MSB1008: Only one project can be specified

[abatishchev]

Task name

DotNetCoreCLI

Task version

2.221.0

Environment type (Please select at least one enviroment where you face this issue)

• [X] Self-Hosted
• [ ] Microsoft Hosted
• [ ] VMSS Pool
• [ ] Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Operation system

OS: Windows Container image: onebranch.azurecr.io/windows/ltsc2019/vse2022:latest

Task log

When works:

Starting: Run tests (windows_build_container)
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version      : 2.221.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
C:\Windows\system32\chcp.com 65001
Active code page: 65001
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
C:\__t\dotnet\dotnet.exe test C:\__w\1\s\src\Service.Tests.Functional\Service.Tests.Functional.csproj --no-restore --no-build --configuration Release --settings C:\__w\1\s\.runsettings --filter TestCategory!=Integration --blame --collect "Code coverage" --logger trx --results-directory C:\__w\_temp\TestResults\ /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=C:\__w\_temp\TestResults\

Welcome to .NET 7.0!
---------------------
SDK Version: 7.0.306

Telemetry
---------
The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
Learn about HTTPS: https://aka.ms/dotnet-https
----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
Test run for C:\__w\1\s\src\Service.Tests.Functional\bin\Release\net7.0\Azure.Throttling.ResourceProvider.Service.Tests.Functional.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.3 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Results File: C:\__w\_temp\TestResults\ContainerAdministrator_6D5F55668976_2023-07-28_22_38_11.trx

Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 3 s - Azure.Throttling.ResourceProvider.Service.Tests.Functional.dll (net7.0)

Url: https://dev.azure.com/msazure/One/_build/results?buildId=77041080&view=logs&j=fa1208be-7322-5e86-1637-d1f5bee81ceb&t=6b44f5bc-7ea1-5a94-5862-d9bc9577db38&l=117

When fails:

Starting: Run tests (windows_build_container)
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version      : 2.221.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
C:\Windows\system32\chcp.com 65001
Active code page: 65001
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
C:\__t\dotnet\dotnet.exe test C:\__w\1\s\src\Service.Tests.Functional\Service.Tests.Functional.csproj --no-restore --no-build --configuration Release --settings C:\__w\1\s\.runsettings --filter TestCategory!=Integration --blame --collect "Code coverage" --logger trx --results-directory C:\__w\_temp\TestResults\ /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=C:\__w\_temp\TestResults\

Welcome to .NET 7.0!
---------------------
SDK Version: 7.0.306

Telemetry
---------
The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
Learn about HTTPS: https://aka.ms/dotnet-https
----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
MSBUILD : error MSB1008: Only one project can be specified.
    Full command line: 'c:\__t\dotnet\sdk\7.0.306\MSBuild.dll -maxcpucount -verbosity:m -target:VSTest -nodereuse:false -nologo -property:VSTestSetting="C:\__w\1\s\.runsettings" -property:VSTestTestCaseFilter="TestCategory!=Integration" -property:VSTestLogger="trx" -property:VSTestNoBuild=true -property:VSTestResultsDirectory="C:\__w\_temp\TestResults\\" -property:VSTestCollect="Code" -property:VSTestBlame=true -property:Configuration=Release C:\__w\1\s\src\Service.Tests.Functional\Service.Tests.Functional.csproj coverage -p:UseSharedCompilation=false -p:EmitCompilerGeneratedFiles=true /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=C:\__w\_temp\TestResults\ -property:VSTestArtifactsProcessingMode=collect -property:VSTestSessionCorrelationId=12004_c1861575-f585-432f-9f5a-74d822feaa59 -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,c:\__t\dotnet\sdk\7.0.306\dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,c:\__t\dotnet\sdk\7.0.306\dotnet.dll'
  Switches appended by response files:
Switch: coverage

For switch syntax, type "MSBuild -help"
##[error]Error: The process 'C:\__t\dotnet\dotnet.exe' failed with exit code 1
C:\__t\dotnet\dotnet.exe test C:\__w\1\s\src\Service.Tests.Integration\Service.Tests.Integration.csproj --no-restore --no-build --configuration Release --settings C:\__w\1\s\.runsettings --filter TestCategory!=Integration --blame --collect "Code coverage" --logger trx --results-directory C:\__w\_temp\TestResults\ /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=C:\__w\_temp\TestResults\
MSBUILD : error MSB1008: Only one project can be specified.
    Full command line: 'c:\__t\dotnet\sdk\7.0.306\MSBuild.dll -maxcpucount -verbosity:m -target:VSTest -nodereuse:false -nologo -property:VSTestSetting="C:\__w\1\s\.runsettings" -property:VSTestTestCaseFilter="TestCategory!=Integration" -property:VSTestLogger="trx" -property:VSTestNoBuild=true -property:VSTestResultsDirectory="C:\__w\_temp\TestResults\\" -property:VSTestCollect="Code" -property:VSTestBlame=true -property:Configuration=Release C:\__w\1\s\src\Service.Tests.Integration\Service.Tests.Integration.csproj coverage -p:UseSharedCompilation=false -p:EmitCompilerGeneratedFiles=true /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=C:\__w\_temp\TestResults\ -property:VSTestArtifactsProcessingMode=collect -property:VSTestSessionCorrelationId=11492_262230af-5788-4b84-920a-ae99d8544e38 -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,c:\__t\dotnet\sdk\7.0.306\dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,c:\__t\dotnet\sdk\7.0.306\dotnet.dll'

Url: https://dev.azure.com/msazure/One/_build/results?buildId=77263266&view=logs&j=15dfcb1a-0989-5cf6-3160-3e181e44de87&t=d80a394b-d4ad-5cf4-e035-192d3dcb341d&l=41

Relevant log output

Dotnet command failed with non-zero exit code on the following projects : [ 'C:\\__w\\1\\s\\src\\Service.Tests.Functional\\Service.Tests.Functional.csproj', 'C:\\__w\\1\\s\\src\\Service.Tests.Integration\\Service.Tests.Integration.csproj' ]

Aditional info

Account: msazure/One Pool: OneBranchPipelines on 1ES Hosted Pool

Task definition:

- task: DotNetCoreCLI@2
  displayName: 'Run tests'
  inputs:
    command: test
    projects: '$(RepoRoot)\src\**\*.Tests.*.csproj'
    arguments: '--no-restore --no-build --configuration ${{ parameters.configuration }} --settings $(RepoRoot)\.runsettings --filter TestCategory!=Integration --blame --collect "Code coverage" --logger "trx" --results-directory $(Agent.TempDirectory)\TestResults\ /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Agent.TempDirectory)\TestResults\'
    publishTestResults: false

[amhokies]

I am seeing a similar problem. It appears to be that --collect "Code coverage" sometimes gets parsed incorrectly and "coverage" ends up as a separate parameter that is interpreted as a second project. Below are the msbuild commands output with detailed verbosity enabled (irrelevant parameters removed):

Successful build:

C:\__t\dotnet\sdk\6.0.412\MSBuild.dll
...
-property:VSTestCollect="Code coverage"
C:\__w\1\s\MySolution.sln

Failed build:

c:\__t\dotnet\sdk\6.0.412\MSBuild.dll 
...
-property:VSTestCollect="Code" 
C:\__w\1\s\MySolution.sln
coverage 

Failures happen seemingly at random with no code changes so I'm a bit perplexed as to what could be causing "Code coverage" to be parsed incorrectly

[v-schhabra]

@abatishchev Can you please add the variable "system.debug" and set the value to "True" in your pipeline and share the success and failure logs?

[abatishchev]

The latest failure: https://dev.azure.com/msazure/One/_build/results?buildId=77731414&view=logs&j=15dfcb1a-0989-5cf6-3160-3e181e44de87&s=6884a131-87da-5381-61f3-d7acc3b91d76&t=45dce97e-2b8e-5d98-6d10-a1ed23901c37&l=41 In a different repo now, what means the issue persists and even spreads.

[abatishchev]

@v-mohithgc I'll try to produce both. The problem is that the failure occurs sporadically.

[K-Cully]

Since last week this has been occurring consistently in some of our builds. (In the Office ADO tenant)

Relevant versions: dotnet ADO task - 2.221.0 Microsoft (R) Test Execution Command Line Tool Version 17.6.3 (x64). Repros with both .NET 7.0.306 and 7.0.400

The below excerpt from the ADO build log shows the issue: *I'm wondering if there is some encoding applied to the "Code coverage" value which leads to it being split, as the command displayed works fine locally.

C:\hostedtoolcache\windows\dotnet\dotnet.exe test D:\a_work\1\s\Omex.sln --logger trx --results-directory D:\a_work_temp -c Release "-p:Platform=Any CPU" --no-build --no-restore --collect "Code coverage" --filter FullyQualifiedName!~CloudTests MSBUILD : error MSB1008: Only one project can be specified. Full command line: 'c:\hostedtoolcache\windows\dotnet\sdk\7.0.306\MSBuild.dll -maxcpucount -verbosity:m -target:VSTest -nodereuse:false -nologo -property:VSTestTestCaseFilter="FullyQualifiedName!~CloudTests" -property:VSTestLogger="trx" -property:VSTestNoBuild=true -property:VSTestResultsDirectory="D:\a_work_temp" -property:VSTestCollect="Code" -property:Configuration=Release D:\a_work\1\s\Omex.sln -p:Platform=Any CPU coverage -p:UseSharedCompilation=false -p:EmitCompilerGeneratedFiles=true -property:VSTestArtifactsProcessingMode=collect -property:VSTestSessionCorrelationId=6036_971960d5-00bc-428a-ac56-b17c8925fee4 -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,c:\hostedtoolcache\windows\dotnet\sdk\7.0.306\dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,c:\hostedtoolcache\windows\dotnet\sdk\7.0.306\dotnet.dll' Switches appended by response files: Switch: CPU

Can share the full build internally, if necessary.

[abatishchev]

@K-Cully are you able to produce a build with debug tracing turned on?

I'm not, I ran 20+ builds manually, all of them have succeeded. Meanwhile automatically triggered builds continue to fail from time to time (across repos, without making any related changes).

[K-Cully]

Yes, it reproduces in one of our pipelines for every run. I'm not going to share the logs here though - The Office org is very sensitive about that sort of thing. Diagnostic logs are available here, if you have access: https://dev.azure.com/office/OC/_build/results?buildId=22098168&view=logs

[AngryBerryMS]

I encountered the same problem, any updates now?

https://dev.azure.com/ceapex/Engineering/_build/results?buildId=1419432&view=logs&j=15dfcb1a-0989-5cf6-3160-3e181e44de87&t=11a9592f-310d-5f1c-c669-7536a224c0e2

[superyyrrzz]

We meet the same error, but in AzureCLI@2 task. It seems that the bug lies in a fundamental component within Azure Pipeline, but is not limited to a specific task.

[justdemon]

We also encounter the same build error. Request to prioritize it. Thanks!

[justdemon]

https://onebranch.visualstudio.com/Support/_workitems/edit/172772

It seems to me it is also a known issue in OneBranch and disabling CodeQL might be a workaround.

[jimmy3912msncom]

We are also seeing the same issue on our side sporadically. It does seem like it is treating the coverage option as a project. I also ask if this can be prioritized.

[amhokies]

We have found another workaround by encoding the space in “Code Coverage” as %20 so that the option is --collect:Code%20coverage

[abatishchev]

@v-mohithgc may I ask what it's waiting for?

[v-mohithgc]

@v-mohithgc may I ask what it's waiting for?

awaiting response from OneBranch team.

[abatishchev]

@v-mohithgc they're offering a workaround of replacing ` with%20` and that's about it. Please escalate.

[Yossibh]

We are facing the same intermittent issue, the %20 workaround worked for us.

[anthony-c-martin]

My team was also broken by this for a number of internal repos.

[abatishchev]

@v-mohithgc please contact the team OneBranch pointed to as the source of the issue - CodeQL.

[c0rn3liusha11]

I am seeing a similar problem. It appears to be that --collect "Code coverage" sometimes gets parsed incorrectly and "coverage" ends up as a separate parameter that is interpreted as a second project. Below are the msbuild commands output with detailed verbosity enabled (irrelevant parameters removed):

Successful build:

C:\__t\dotnet\sdk\6.0.412\MSBuild.dll
...
-property:VSTestCollect="Code coverage"
C:\__w\1\s\MySolution.sln

Failed build:

c:\__t\dotnet\sdk\6.0.412\MSBuild.dll 
...
-property:VSTestCollect="Code" 
C:\__w\1\s\MySolution.sln
coverage 

Failures happen seemingly at random with no code changes so I'm a bit perplexed as to what could be causing "Code coverage" to be parsed incorrectly

We are seeing this happen in many pipelines in dev.azure.com/microsoft tenant as well. The logging from the task seems to indicate that the issue is actually in the dotnet test code as the commandline that is logged for the dotnet test command appears correct, but the subsequent (transformed) commandline that runs msbuild is where the parsing error has occurred.

We have tried multiple different options for passing in the --collect argument, none of which have been consistently successful. {--collect:"Code coverage"; --collect "Code coverage"; etc.}. IMO the bug is in the parsing/transformation when the value is quoted and includes a space, but that is only hypothesis.

[abatishchev]

Can everyone confirm it stopped failing for me? The CodeQL reportingly has fixed the issue.

[v-mohithgc]

The CodeOL team has reported this issue has been fixed, will be closing this issue for now, please feel free to reopen if any recent occurrence found. Thanks