search

dotnet / source-build

A repository to track efforts to produce a source tarball of the .NET Core SDK and all its components
MIT License
265 stars 130 forks source link

Smoke tests failing in stage 2 with `VSTESTTASK2` error #4003

Closed mthalman closed 7 months ago

mthalman commented 8 months ago 
VSTESTTASK2 : error Message:  [/vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/Microsoft.DotNet.SourceBuild.SmokeTests.csproj]

No other information exists. It's an error contained in the build output but not an actual test case failure. Example build (internal link)

It would seem this is introduced by new functionality from within vstest that gets compiled into the SDK and used in the stage 2 build.

mthalman commented 8 months ago

This may be a regression caused by https://github.com/microsoft/vstest/pull/4834. These tests were working last Friday and failing today after incorporating changes that included https://github.com/microsoft/vstest/pull/4834. @nohwnd - Can you take a look? Here's a binlog: BuildTests_0122193259.zip. We need to get this resolved soon for .NET 9 Preview 1.

mthalman commented 8 months ago

I ran a test build (internal link) that reverted the change from https://github.com/microsoft/vstest/pull/4834 but that didn't help at all.

nohwnd commented 8 months ago

This is more likely linked to this change: https://github.com/dotnet/sdk/pull/38098/files which changes how failed tests are reported to take advantage of msbuild terminal logger which is now the default experience for dotnet build.

I am trying to repro to see if the smoke tests failed and VSTEST fails to report the details of the failure. Or if VSTEST is failing and the change needs to be reverted.

nohwnd commented 8 months ago

Running another test build with opt-out from the new feature https://dev.azure.com/dnceng/internal/_build/results?buildId=2360131&view=results

In the meantime I am trying to repro locally.

nohwnd commented 8 months ago

I can't repro fully.

I can run the tests but I always get the full output of the error. And I can't run tests without failures locally because I don't have all the artifacts. I always see: Tarball path '' specified in does not exist. ( from here ) I downloaded what I could but I still cannot get it to work locally.

I can see in binary logger the same error output as in build, but it is followed by the actual error message.

In your run I cannot see any failures in trx, so I am not sure where the errors are coming from and if they are real.

I've edited the target to actually propagate the opt-out flag and started another build here (from a new branch, I want to see how the other one will finish) https://dev.azure.com/dnceng/internal/_build/results?buildId=2360247&view=results

MichaelSimons commented 7 months ago

[Triage] closing as is appears this fix flowed in.