System.Object (and other types) not defined when running tests (missing assembly references)

[bluetarpmedia]

Bug description

On a Windows 10 PC with VS 2019 16.9.4 and .NET SDK 5.0.202 installed, I checked out the latest commit (ad578dfef652387364f3ec2a3840596a4449dc0c as of 10-May-21), ran the init.ps1 script, then dotnet build (0 warnings, 0 errors), and then dotnet test. This produced many errors like this:

error CS0012: The type 'Object' is defined in an assembly that is not referenced.
You must add a reference to assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.

(And variations of the above for System.Void, System.Int32, System.Type, etc) and:

error CS0234: The type or namespace name 'Task' does not exist in the namespace 'System.Threading.Tasks' (are you missing an assembly reference?)
DiagnosticResult.CompilerError("CS0234").WithSpan(6, 37, 6, 41).WithArguments("Task", "System.Threading.Tasks"),

Total: Failed: 81, Passed: 1, Skipped: 0, Total: 82, Duration: 54 s

(The one test that passes is AssemblyHasNoReferenceToMpf.)

See screenshot below for more details.

To Reproduce

1. Checkout latest ad578dfef652387364f3ec2a3840596a4449dc0c
2. Open Powershell prompt, run the init.ps1 script
3. Then run dotnet build
4. Then run dotnet test (or dotnet test -f net472)

Or after step 2, open the solution in Visual Studio 2019 16.9.4 and then build, then run the tests in the Test Explorer.

Expected behavior

The tests should all pass (according to the Azure builds).

More info

I can get most of the tests to pass by explicitly adding metadata references in the Test ctor (CSharpCodeFixVerifier`2.cs line 77), like this:

this.SolutionTransforms.Add((solution, projectId) =>
{
    Project? project = solution.GetProject(projectId);

    project = project?.AddMetadataReference(MetadataReference.CreateFromFile(typeof(object).Assembly.Location));
    project = project?.AddMetadataReference(MetadataReference.CreateFromFile(typeof(System.ComponentModel.Design.IServiceContainer).Assembly.Location));
    project = project?.AddMetadataReference(PresentationFrameworkReference);

    if (project == null)
    {
        throw new System.NullReferenceException(nameof(project));
    }

    return project.Solution;
});

But some of the VSSDK006 tests still fail with this error:

error CS0012: The type 'Object' is defined in an assembly that is not referenced. You must add a reference to assembly 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
DiagnosticResult.CompilerError("CS0012").WithSpan(11, 17, 11, 24).WithArguments("System.Object", "netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51"),

It's unclear why some VSSDK006 tests pass and others don't. E.g. LocalAssigned_GetService_ThenUsedAsync fails with that error, but LocalAssigned_GetService_ThenUsed_WithNullConditionalAsync passes.

New total with above workaround: Failed: 17, Passed: 65, Skipped: 0, Total: 82, Duration: 1 m 4 s

Screenshots

[bluetarpmedia]

The same steps on a different Windows 10 PC (same VS 2019 and .NET SDK version) worked fine.

[AArnott]

Our arbiter in situations like these is the Azure Pipelines hosted agent. If it works there (and presumably some other dev boxes) then it's good and the broken box should investigate what's wrong. However if you find out what's "wrong" with the broken box and feel it's something this repo should handle better and can send a PR, we might entertain that.

[bluetarpmedia]

Yeah, it's definitely some kind of configuration problem on that particular computer. If I find the problem I'll try to remember to post an update here for anybody in the future.