search

dotnet / roslyn

The Roslyn .NET compiler provides C# and Visual Basic languages with rich code analysis APIs.
https://docs.microsoft.com/dotnet/csharp/roslyn-sdk/
MIT License
19.07k stars 4.04k forks source link

CI system should print out integration test failures #54843

Open CyrusNajmabadi opened 3 years ago

CyrusNajmabadi commented 3 years ago

Right now all you see are things like: 

Proc dump location: D:\workspace\_work\1\s\.tools\ProcDump\procdump.exe 
Running 2 test assemblies in 2 partitions 
 1 running, 1 queued, 0 completed 
Roslyn Error: test timeout exceeded, dumping remaining processes 
Dumping dotnet 5796 to D:\workspace\_work\1\s\artifacts\log\Debug\dotnet-1.dmp ... succeeded (100469002 bytes) 
Dumping dotnet 5744 to D:\workspace\_work\1\s\artifacts\log\Debug\dotnet-2.dmp ... succeeded (105067654 bytes) 
Dumping testhost.net472.x86 2520 to D:\workspace\_work\1\s\artifacts\log\Debug\testhost.net472.x86-3.dmp ... succeeded (325525651 bytes) 
Copying ServiceHub logs to D:\workspace\_work\1\s\artifacts\log\Debug 
##[error](NETCORE_ENGINEERING_TELEMETRY=Test) Tests failed

This is cryptic and hard to diagnose. Can we instead print out the test log failure info? It looks like we have it somewhere, so it would just be about printing it to the console.

RikkiGibson commented 3 years ago

Example run where the failure is hard to diagnose: https://dev.azure.com/dnceng/public/_build/results?buildId=1238413&view=logs&j=cf757b30-6920-5b57-d4b8-8b599fbc5c5f&t=2213ea15-66b2-5fe4-0869-050835bf49d2

sharwell commented 3 years ago

This type of failure is diagnosed by downloading the heap dump and finding the deadlock. Historically, test logs have not made the investigation any easier so the emphasis is on ensuring the heap dump is available.

CyrusNajmabadi commented 3 years ago

I have run into this many times before and it's an utter PITA. Please provide just the test that was running. We can still keep providing the heap dumps if hte dev needs that.