search

dotnet / templating

This repo contains the Template Engine which is used by dotnet new
https://docs.microsoft.com/dotnet/
MIT License
1.59k stars 365 forks source link

Inconsistent test failure in ValidateCommand_BasicTest #8121

Open MiYanni opened 1 month ago

MiYanni commented 1 month ago

Summary

I've seen this test fail a few times in the last few days of reviewing PRs. Here, it failed on the Windows Release job:

VerifyException : Directory: D:\a\_work\1\s\test\Microsoft.TemplateEngine.Authoring.CLI.IntegrationTests\Snapshots
NotEqual:
  - Received: ValidateCommandTests.ValidateCommand_BasicTest.Windows.received.txt
    Verified: ValidateCommandTests.ValidateCommand_BasicTest.Windows.verified.txt

FileContent:

NotEqual:

Received: ValidateCommandTests.ValidateCommand_BasicTest.Windows.received.txt
Verified: ValidateCommandTests.ValidateCommand_BasicTest.Windows.verified.txt
Compare Result:
54       Template 'name' (TestAssets.Invalid.Localization.ValidationFailure): the template is valid.
 -       'de-DE' localization for the template 'name' (TestAssets.Invalid.Localization.ValidationFailure): the localization file is not valid. The localization will be skipped.
 +       'de-DE' localization for the template 'name' (TestAssets.
 -       'tr' localization for the template 'name' (TestAssets.Invalid.Localization.ValidationFailure): the localization file is not valid. The localization will be skipped.
 -       
   [EOF]

Seems to have to be some kind of localization error. Not sure why it isn't consistently failing.

MiYanni commented 1 month ago

Another one on Windows Debug job:

VerifyException : Directory: D:\a\_work\1\s\test\Microsoft.TemplateEngine.Authoring.CLI.IntegrationTests\Snapshots
NotEqual:
  - Received: ValidateCommandTests.ValidateCommand_BasicTest.Windows.received.txt
    Verified: ValidateCommandTests.ValidateCommand_BasicTest.Windows.verified.txt

FileContent:

NotEqual:

Received: ValidateCommandTests.ValidateCommand_BasicTest.Windows.received.txt
Verified: ValidateCommandTests.ValidateCommand_BasicTest.Windows.verified.txt
Compare Result:
54       Template 'name' (TestAssets.Invalid.Localization.ValidationFailure): the template is valid.
 -       'de-DE' localization for the template 'name' (TestAssets.Invalid.Localization.ValidationFailure): the localization file is not valid. The localization will be skipped.
 +       'de-DE' localization for the template 'name' (TestAssets.
 -       'tr' localization for the template 'name' (TestAssets.Invalid.Localization.ValidationFailure): the localization file is not valid. The localization will be skipped.
 -       
   [EOF]