microsoft / vstest

Visual Studio Test Platform is the runner and engine that powers test explorer and vstest.console.
MIT License
897 stars 322 forks source link

dotnet msbuild /t:VSTest does not work with terminal logger #5156

Open nohwnd opened 3 months ago

nohwnd commented 3 months ago
dotnet msbuild /t:VSTest /v:n

Does not work with terminal logger and node reuse. It will not print the error from the second test project.


The following reproduces it for me:

mkdir Project1
cd Project1
dotnet new xunit
cd ..

mkdir Project2
cd Project2
dotnet new xunit
cd ..

dotnet new sln
dotnet sln add Project1\Project1.csproj
dotnet sln add Project2\Project2.csproj

Add the following to Project2\UnitTest1.cs:

    Assert.Equal(1, 2);

Then:

dotnet msbuild /t:VSTest /v:n

I 'think' due to the parallel nature of MSBuild, it can vary which project you have to add the failing test to, but this gives me the following output:

C:\Temp\vstest1503>dotnet msbuild /t:VSTest /v:n
Microsoft (R) Build Engine version 15.7.179.6572 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 06/08/2018 09:17:45.
     1>Project "C:\Temp\vstest1503\vstest1503.sln" on node 1 (VSTest target(s)).
Build started, please wait...
     1>Project "C:\Temp\vstest1503\vstest1503.sln" (1) is building "C:\Temp\vstest1503\Project2\Project2.csproj" (2) on node 2 (VSTest target(s)).
     2>Project "C:\Temp\vstest1503\Project2\Project2.csproj" (2) is building "C:\Temp\vstest1503\Project2\Project2.csproj" (2:2) on node 2 (default targets).
     2>GenerateTargetFrameworkMonikerAttribute:
       Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
       CoreGenerateProgramFile:
       Skipping target "CoreGenerateProgramFile" because all output files are up-to-date with respect to the input files.
       CoreGenerateAssemblyInfo:
       Skipping target "CoreGenerateAssemblyInfo" because all output files are up-to-date with respect to the input files.
       CoreCompile:
       Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
       _CopyOutOfDateSourceItemsToOutputDirectory:
       Skipping target "_CopyOutOfDateSourceItemsToOutputDirectory" because all output files are up-to-date with respect to the input files.
       GenerateBuildDependencyFile:
       Skipping target "GenerateBuildDependencyFile" because all output files are up-to-date with respect to the input files.
       GenerateBuildRuntimeConfigurationFiles:
       Skipping target "GenerateBuildRuntimeConfigurationFiles" because all output files are up-to-date with respect to the input files.
       CopyFilesToOutputDirectory:
         Project2 -> C:\Temp\vstest1503\Project2\bin\Debug\netcoreapp2.1\Project2.dll
     2>Done Building Project "C:\Temp\vstest1503\Project2\Project2.csproj" (default targets).
     1>Project "C:\Temp\vstest1503\vstest1503.sln" (1) is building "C:\Temp\vstest1503\Project1\Project1.csproj" (3) on node 1 (VSTest target(s)).
     3>Project "C:\Temp\vstest1503\Project1\Project1.csproj" (3) is building "C:\Temp\vstest1503\Project1\Project1.csproj" (3:2) on node 1 (default targets).
     3>GenerateTargetFrameworkMonikerAttribute:
       Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
       CoreGenerateProgramFile:
       Skipping target "CoreGenerateProgramFile" because all output files are up-to-date with respect to the input files.
       CoreGenerateAssemblyInfo:
       Skipping target "CoreGenerateAssemblyInfo" because all output files are up-to-date with respect to the input files.
       CoreCompile:
       Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
       _CopyOutOfDateSourceItemsToOutputDirectory:
       Skipping target "_CopyOutOfDateSourceItemsToOutputDirectory" because all output files are up-to-date with respect to the input files.
       GenerateBuildDependencyFile:
       Skipping target "GenerateBuildDependencyFile" because all output files are up-to-date with respect to the input files.
       GenerateBuildRuntimeConfigurationFiles:
       Skipping target "GenerateBuildRuntimeConfigurationFiles" because all output files are up-to-date with respect to the input files.
       CopyFilesToOutputDirectory:
         Project1 -> C:\Temp\vstest1503\Project1\bin\Debug\netcoreapp2.1\Project1.dll
     3>Done Building Project "C:\Temp\vstest1503\Project1\Project1.csproj" (default targets).
Build completed.

Test run for C:\Temp\vstest1503\Project1\bin\Debug\netcoreapp2.1\Project1.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 15.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
     2>Done Building Project "C:\Temp\vstest1503\Project2\Project2.csproj" (VSTest target(s)) -- FAILED.

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 2.3674 Seconds
     3>Done Building Project "C:\Temp\vstest1503\Project1\Project1.csproj" (VSTest target(s)).
     1>Done Building Project "C:\Temp\vstest1503\vstest1503.sln" (VSTest target(s)) -- FAILED.

Build FAILED.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:03.71

Note, build failed, but no output of Project2 tests.

Originally posted by @FlukeFan in https://github.com/microsoft/vstest/issues/1503#issuecomment-410627578

FlukeFan commented 3 months ago

I hadn't realised that #1503 had fixed it for dotnet test ... looking at the implementation here: https://github.com/dotnet/sdk/blob/v9.0.100-preview.6.24328.19/src/Cli/dotnet/commands/dotnet-test/Program.cs#L72

it looks like the following works: dotnet msbuild /t:VSTest /p:VsTestUseMSBuildOutput=true

mcartoixa commented 3 months ago

Yep. IMHO msbuild should not have to specify that it wants msbuild to be compatible with msbuild. Defaults should be flipped.