Package reference alias tests are disabled

[dsplaisted]

The tests in GivenThatWeHaveAPackageReferenceWithAliases have started failing somewhat consistently in CI. I haven't been able to reproduce the issue locally. The failures look something like this:

/private/tmp/helix/working/B27409DB/w/A434093A/e/testExecutionDirectory/CanBuildProje---B79B2BCA/Project/Project.cs(9,13): error CS0118: 'First.A' is a namespace but is used like a type [/private/tmp/helix/working/B27409DB/w/A434093A/e/testExecutionDirectory/CanBuildProje---B79B2BCA/Project/Project.csproj]
/private/tmp/helix/working/B27409DB/w/A434093A/e/testExecutionDirectory/CanBuildProje---B79B2BCA/Project/Project.cs(10,20): error CS0234: The type or namespace name 'B' does not exist in the namespace '<global namespace>' (are you missing an assembly reference?) [/private/tmp/helix/working/B27409DB/w/A434093A/e/testExecutionDirectory/CanBuildProje---B79B2BCA/Project/Project.csproj]

This seems like it's probably a test issue, perhaps related to the fact that these tests create and then consume NuGet packages within the test. Because of this, I am going to disable these tests for now. To further investigate, we probably need to figure out how to get a binlog of the failures in Helix.

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[dsplaisted]

FYI @jaredpar @dotnet/nuget-team I'm disabling tests for using aliases with package references. I'm pretty sure this was a test issue but keep an eye out, it's possible this could have been an issue in NuGet, the compiler, or the .NET SDK.

[jaredpar]

Do we have tests in NuGet that we can think of as sufficient coverage here? Also is the binlog available when these tests fail?

[aortiz-msft]

@jaredpar - Here's aliases tests in NuGet.Client:

1. RestoreNetCore_PackageReferenceWithAliases_IsReflectedInAssetsFileAsync()
2. RestoreNetCore_WithCustomAliases_WritesConditionWithCorrectAlias()
3. NominateProjectAsync_PackageReferenceWithAliases(bool isV2Nominate)
4. DotnetRestore_PackageReferenceWithAliases_ReflectedInTheAssetsFile()
5. DotnetRestore_MultiTargettingWithAliases_Succeeds()
6. RestoreCommand_WhenPackageReferenceHasAliases_IsReflectedInTheCompileTimeAssemblies()
7. GetTargetFrameworkInfos_WithCustomAliases_InfersCorrectTargetFramework()
8. PackageSpecReader_ReadsRestoreMetadataWithAliases(IEnvironmentVariableReader environmentVariableReader)
9. RoundTripPackageReferenceAliases()
10. RoundTripTargetFrameworkAliases()

I don't know how these compare to the ones in the SDK repo, but disabling tests is scary.

@dsplaisted: Have you ensured that the tests run sequentially to discard parallelization issues?

cc @zivkan, @nkolev92

[nkolev92]

NuGet's alias test also only test the NuGet side of applying the aliases, ie we check the assets file. We don't really run a build, since that requires the full SDK, and we don't always have the latest version. I think it'd be great to keep the SDK side tests.