search

dotnet / sdk

Core functionality needed to create .NET Core projects, that is shared between Visual Studio and CLI
https://dot.net/core
MIT License
2.59k stars 1.03k forks source link

Reenable templating tests for windows #41795

Open marcpopMSFT opened 1 week ago

marcpopMSFT commented 1 week ago

https://github.com/dotnet/sdk/pull/41794

The templating tests appear to have individual tests that are left running for hours and eventually time out. As far as I can tell, any test that runs dotnet new can hit this so there may be a hang in the template engine or there could be a problem with the test. Since we don't know what's going on, it's better to disable the tests for now than continue to fight this.

Pipelines - Run 20240624.19 (azure.com) Pipelines - Run 20240621.47 (azure.com)

dotnet-new.IntegrationTests: [Long Running Test] 'Microsoft.DotNet.Cli.New.IntegrationTests.DotnetNewListTests.CanFilterTags_WithNameCriteria', Elapsed: 02:44:23

dotnet-new.IntegrationTests: [Long Running Test] 'Microsoft.DotNet.Cli.New.IntegrationTests.DotnetNewInstantiateTests.CanInstantiateTemplate_WithConditions_BasedOnFileName', Elapsed: 02:39:08 [Long Running Test] 'Microsoft.DotNet.Cli.New.IntegrationTests.PostActionTests.PrintInstructions_Basic_Approval', Elapsed: 02:08:05 [Long Running Test] 'Microsoft.DotNet.Cli.New.IntegrationTests.TemplateEngineSamplesTest.TemplateEngineSamplesProjectTest', Elapsed: 02:44:16 [Long Running Test] 'Microsoft.DotNet.Cli.New.IntegrationTests.DotnetNewListTests.TemplateWithSpecifiedColumnOutput', Elapsed: 02:44:16

dotnet-new.IntegrationTests: [Long Running Test] 'Microsoft.DotNet.Cli.New.IntegrationTests.DotnetNewDebugOptionsTests.CanShowBasicInfoWithDebugRebuildCache', Elapsed: 02:35:33 [Long Running Test] 'Microsoft.DotNet.Cli.New.IntegrationTests.CommonTemplatesTests.AllCommonItemsCreate', Elapsed: 02:35:32

marcpopMSFT commented 1 week ago

Here's my investigation so far: The first build I could find that timed out in the templating leg was on 6/18 in the runtime PR: Pipelines - Run 20240618.45 (azure.com) I don't see any time out builds from that leg for the prior 2 weeks so I think this was the initial one.

The next one I see timing out happened after this PR got merged on the 20th and then we see multiple a day after that. It's possible some other change went in that this PR picked up but given how common this became on the 20th, I'm inclined to believe the runtime PR caused this. I don't see any other PR builds failing on the 19th which also supports my guess that the runtime caused this.

Query failed builds: Pipelines - Runs for dotnet-sdk-public-ci (azure.com)

Query pull requests around the same time: Pull requests · dotnet/sdk (github.com)

PR I believe is problematic: [main] Update dependencies from dotnet/runtime by dotnet-maestro[bot] · Pull Request #41616 · dotnet/sdk (github.com)

Runtime build with the first timeout: 9.0.0-preview.6.24318.5

There were multiple runtime builds before this in that PR that didn't have the timeout including ones on the 16th so I'm thinking something checked into runtime between the 16th and 18th.