search

dotnet / roslyn

The Roslyn .NET compiler provides C# and Visual Basic languages with rich code analysis APIs.
https://docs.microsoft.com/dotnet/csharp/roslyn-sdk/
MIT License
19.03k stars 4.03k forks source link

NewlyCreatedProjectsFromDotNetNew.ValidateCSharpTemplateProjects is flakey #74157

Open CyrusNajmabadi opened 4 months ago

CyrusNajmabadi commented 4 months ago

Failed https://dev.azure.com/dnceng-public/public/_build/results?buildId=720157&view=ms.vss-test-web.build-test-results-tab&runId=18060982&resultId=100226&paneView=debug

Expected 0 items but found 17: The following compiler diagnostics are being reported for the template.
Items:
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\obj\Debug\net9.0\65921193-f413-4206-afc8-9657782aa1d4.GlobalUsings.g.cs(2,32): error CS0234: The type or namespace name 'Playwright' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\obj\Debug\net9.0\65921193-f413-4206-afc8-9657782aa1d4.GlobalUsings.g.cs(3,22): error CS0400: The type or namespace name 'NUnit' could not be found in the global namespace (are you missing an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(5,22): error CS0246: The type or namespace name 'PageTest' could not be found (are you missing a using directive or an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(3,2): error CS0246: The type or namespace name 'ParallelizableAttribute' could not be found (are you missing a using directive or an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(3,2): error CS0246: The type or namespace name 'Parallelizable' could not be found (are you missing a using directive or an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(4,2): error CS0246: The type or namespace name 'TestFixtureAttribute' could not be found (are you missing a using directive or an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(4,2): error CS0246: The type or namespace name 'TestFixture' could not be found (are you missing a using directive or an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(3,17): error CS0103: The name 'ParallelScope' does not exist in the current context
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(7,6): error CS0246: The type or namespace name 'TestAttribute' could not be found (are you missing a using directive or an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(7,6): error CS0246: The type or namespace name 'Test' could not be found (are you missing a using directive or an assembly reference?)
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(10,15): error CS0103: The name 'Page' does not exist in the current context
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(13,15): error CS0103: The name 'Expect' does not exist in the current context
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(13,22): error CS0103: The name 'Page' does not exist in the current context
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(16,26): error CS0103: The name 'Page' does not exist in the current context
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(19,15): error CS0103: The name 'Expect' does not exist in the current context
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(25,15): error CS0103: The name 'Expect' does not exist in the current context
T:\RoslynTests\65921193-f413-4206-afc8-9657782aa1d4\UnitTest1.cs(25,22): error CS0103: The name 'Page' does not exist in the current context
dotnet-issue-labeler[bot] commented 4 months ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

dotnet-issue-labeler[bot] commented 4 months ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

RikkiGibson commented 3 months ago

@jjonescz I seem to recall you filing a bug on a template author recently? Is this issue related to that?

jjonescz commented 3 months ago

I seem to recall you filing a bug on a template author recently? Is this issue related to that?

Yes: https://github.com/dotnet/test-templates/issues/412 I have even disabled the failing playwright test scenario in https://github.com/dotnet/roslyn/pull/73656/commits/9acbcf77dd9351ea0bff8707e8214f1b4e19fd1d but perhaps that stopped working because I see the whole theory is now disabled against this issue

https://github.com/dotnet/roslyn/blob/a6824ae36db3352538fd4892dbb11325c73d40d0/src/Workspaces/MSBuildTest/NewlyCreatedProjectsFromDotNetNew.cs#L64

jjonescz commented 3 months ago

Anyway that bug should be fixed in .NET 9 preview 6 which should flow in through arcade soon I think.