Add Razor and workload template tests

[mattscheffer]

Todo: Find a better method to fail gracefully rather than throwing an exception if workload list doesn't show the expected workload is installed/uninstalled

[mattscheffer]

@mmitche Do you know if there are any restrictions on installing/uninstalling workloads (using dotnet workload) on the build machines? I'm surprised to see windows is failing here while Linux/Mac are passing.

[mmitche]

@mmitche Do you know if there are any restrictions on installing/uninstalling workloads (using dotnet workload) on the build machines? I'm surprised to see windows is failing here while Linux/Mac are passing.

I don't think there should be an issue installing the workloads. The machines are fresh every time

[mattscheffer]

@mmitche I'm not sure if this is using a different install location or we're actually unable to uninstall workloads on the Windows machines. Output Logs indicate there isn't any problem uninstalling workloads, but dotnet workload list still shows wasm-tools in addition to 4 other workloads. Mac/Linux don't show any other workloads installed curiously

dotnet workload uninstall wasm-tools Removing workload installation record for wasm-tools...Garbage collecting for SDK feature band(s) 9.0.100-alpha.1... Uninstalling workload pack Microsoft.NET.Runtime.Emscripten.Cache version 9.0.0-alpha.1.24061.1... Uninstalling workload pack Microsoft.NET.Runtime.Emscripten.Node version 9.0.0-alpha.1.24061.1... Uninstalling workload pack Microsoft.NET.Runtime.Emscripten.Python version 9.0.0-alpha.1.24061.1... Uninstalling workload pack Microsoft.NET.Runtime.Emscripten.Sdk version 9.0.0-alpha.1.24061.1...Uninstalling workload pack Microsoft.NET.Runtime.MonoAOTCompiler.Task version 9.0.0-alpha.1.24062.4... Uninstalling workload pack Microsoft.NET.Runtime.MonoTargets.Sdk version 9.0.0-alpha.1.24062.4... Uninstalling workload pack Microsoft.NET.Runtime.WebAssembly.Sdk version 9.0.0-alpha.1.24062.4... Uninstalling workload pack Microsoft.NETCore.App.Runtime.AOT.Cross.browser-wasm version 9.0.0-alpha.1.24062.4... Uninstalling workload pack Microsoft.NETCore.App.Runtime.Mono.browser-wasm version 9.0.0-alpha.1.24062.4... Successfully uninstalled workload(s): wasm-tools

windows - dotnet workload list Installed Workload Id Manifest Version Installation Source----------------------------------------------------------------------------------------- maui-windows 9.0.0-ci.net9.9858/9.0.100-alpha.1 VS 17.8.34330.188
maccatalyst 17.0.9804-net9-p1/9.0.100-alpha.1 VS 17.8.34330.188 ios 17.0.9804-net9-p1/9.0.100-alpha.1 VS 17.8.34330.188
android 34.99.0-preview.1.120/9.0.100-alpha.1 VS 17.8.34330.188
wasm-tools 9.0.0-alpha.1.24062.4/9.0.100-alpha.1 VS 17.8.34330.188
Use dotnet workload search to find additional workloads to install.

[mmitche]

I wonder if those are coming in via VS...which dotnet installation is beign used when uninstalling the workloads? I don't know a ton about how the workload installation works. What I would hope is that workloads aren't globally installed. i.e. we can have two dotnet installations on the machine, and both can install the same workloads.

@marcpopMSFT @dsplaisted @joeloff any ideas?

[mattscheffer]

I wonder if those are coming in via VS...which dotnet installation is beign used when uninstalling the workloads? I don't know a ton about how the workload installation works. What I would hope is that workloads aren't globally installed. i.e. we can have two dotnet installations on the machine, and both can install the same workloads.

@marcpopMSFT @dsplaisted @joeloff any ideas?

At least in my local tests, workloads installed via these tests aren't globally installed. IDK about VS either.

[dsplaisted]

I wonder if those are coming in via VS...which dotnet installation is beign used when uninstalling the workloads? I don't know a ton about how the workload installation works. What I would hope is that workloads aren't globally installed. i.e. we can have two dotnet installations on the machine, and both can install the same workloads.

@marcpopMSFT @dsplaisted @joeloff any ideas?

The rightmost column shows that those workloads are coming from VS. When the SDK is installed via MSI (which is the case with either the standalone installer on Windows or via VS), the SDK is in a sense globally installed and so are the workloads. But you can have multiple versions of the SDK installed side-by-side, and they can each have different workloads installed.

However, if VS has installed the same feature band of the SDK as the standalone installer, then workloads installed by VS will be available on the command-line and vice versa. Probably in this case the test should check to see what workloads are installed before it does anything, and then when it's done and uninstalled workloads check that the same set of workloads are installed as were before the test ran (rather than testing that none are installed).

Also, I'd recommend using dotnet workload list --machine-readable to get an output that's easier to parse.

[mattscheffer]

The rightmost column shows that those workloads are coming from VS. When the SDK is installed via MSI (which is the case with either the standalone installer on Windows or via VS), the SDK is in a sense globally installed and so are the workloads. But you can have multiple versions of the SDK installed side-by-side, and they can each have different workloads installed.

However, if VS has installed the same feature band of the SDK as the standalone installer, then workloads installed by VS will be available on the command-line and vice versa. Probably in this case the test should check to see what workloads are installed before it does anything, and then when it's done and uninstalled workloads check that the same set of workloads are installed as were before the test ran (rather than testing that none are installed).

Also, I'd recommend using dotnet workload list --machine-readable to get an output that's easier to parse.

Thanks! I see the spots you're talking about here. After an expanded look at the 'lists', workloads are uninstalling the sdk source correctly with just the VS source remaining. I essentially just need to be pickier with the string output checks. Will try out --machine-readable as well.

[dsplaisted]

Note that --machine-readable doesn't currently tell you the installation source. We could add that in the future if needed. I also think that its output format is going to change in .NET 9 so that it only outputs the JSON, not anything else (and not including the ==workloadListJsonOutputStart== and ==workloadListJsonOutputEnd== lines.

[mattscheffer]

Ah ok. Well in that case, we'd just run right back into the same problem again. It'd work for linux/mac I assume, but not windows and we'd want to keep that uniform.

Thanks for the heads up about the output change!

[mattscheffer]

@dsplaisted I did a comparison as suggested before and after install/uninstall. I got wildly different manifest versions between them which is quite surprising to see as again the only thing done in the middle of this was working with the wasm-tools workload. Is 'dotnet workload list' only capable of grabbing 1-2 versions at a time and there are just a number of versions installed on these machines? Do you know the logic behind what it picks?

Before workload install Installed Workload Id Manifest Version Installation Source

---

maui-windows 9.0.0-ci.net9.9866/9.0.100-alpha.1 VS 17.8.34408.163
maccatalyst 17.2.9035-net9-p1/9.0.100-alpha.1 VS 17.8.34408.163
ios 17.2.9035-net9-p1/9.0.100-alpha.1 VS 17.8.34408.163
android 34.99.0-preview.1.129/9.0.100-alpha.1 VS 17.8.34408.163
wasm-tools 9.0.0-alpha.1.24066.33/9.0.100-alpha.1 VS 17.8.34408.163
Use dotnet workload search to find additional workloads to install.

After workload uninstall Installed Workload Id Manifest Version Installation Source

---

maui-windows 8.0.0-rc.1.9171/8.0.100-rc.1 VS 17.8.34408.163
maccatalyst 16.4.8825-net8-rc1/8.0.100-rc.1 VS 17.8.34408.163
ios 16.4.8825-net8-rc1/8.0.100-rc.1 VS 17.8.34408.163
android 34.0.0-rc.1.432/8.0.100-rc.1 VS 17.8.34408.163
wasm-tools 9.0.0-alpha.1.23614.10/9.0.100-alpha.1 VS 17.8.34408.163
Use dotnet workload search to find additional workloads to install.

[dsplaisted]

When you install a workload, by default the CLI will first update to the latest workload versions. That is probably why the versions are different before and after you do the install/uninstall. You can prevent the version update with the --skip-manifest-update option on install.

[mattscheffer]

When you install a workload, by default the CLI will first update to the latest workload versions. That is probably why the versions are different before and after you do the install/uninstall. You can prevent the version update with the --skip-manifest-update option on install.

Thanks! That got it fixed so it's working and passing now.

@mmitche @joeloff can you review when you have a chance. Richa mentioned that I should add you as it's adding a test for dotnet workload.

[mattscheffer]

@mmitche Would you mind reviewing/pushing when you have a chance?

[mattscheffer]

Or @tkapin if you're available

[tkapin]

Or @tkapin if you're available

Approved with few suggestions.

[tkapin]

One more general comment, consider using logger instead of Console.WriteLine(). Not a hard requirement, but it might simplify some things later, such as logging debug messages with different verbosity etc.

[mattscheffer]

Thanks for reviewing Tomas!

I'll look into making the logger change. Having extra verbosity does sound ideal, however I think those should go will go in a different PR. I think there are a couple of other places using Console.WriteLine and I'd rather change them all at once.

[mmitche]

@tkapin Feel free to merge if you think this is ready.