dotnet / sdk

Core functionality needed to create .NET Core projects, that is shared between Visual Studio and CLI
https://dot.net/core
MIT License
2.65k stars 1.05k forks source link

[9.0-preview.7] Microsoft.DotNet.MSBuildWorkloadSdkResolver fails with dotnet workload or build on Windows #42211

Open martincostello opened 1 month ago

martincostello commented 1 month ago

Describe the bug

Somewhere between 9.0.100-preview.7.24365.2 and 9.0.100-preview.7.24365.10 (diff) of the .NET SDK, something has started to cause dotnet workload restore and dotnet build to fail with the following exception on Windows (but not on Linux or macOS):

Unhandled exception: Microsoft.Build.Exceptions.InvalidProjectFileException: SDK Resolver Failure: "The SDK resolver "Microsoft.DotNet.MSBuildWorkloadSdkResolver" failed while attempting to resolve the SDK "Microsoft.NET.SDK.WorkloadAutoImportPropsLocator". Exception: "System.IO.FileNotFoundException: Workload manifest microsoft.net.sdk.aspire: 8.0.1/8.0.100 from workload version 9.0.100-preview.7.24367.5 was not installed. Running "dotnet workload repair" may resolve this.
   at Microsoft.NET.Sdk.WorkloadManifestReader.SdkDirectoryWorkloadManifestProvider.GetManifests()
   at Microsoft.NET.Sdk.WorkloadManifestReader.WorkloadResolver.LoadManifestsFromProvider(IWorkloadManifestProvider manifestProvider)
   at Microsoft.NET.Sdk.WorkloadManifestReader.WorkloadResolver.InitializeManifests()
   at Microsoft.NET.Sdk.WorkloadManifestReader.WorkloadResolver.GetInstalledWorkloadPacksOfKind(WorkloadPackKind kind)+MoveNext()
   at Microsoft.NET.Sdk.WorkloadMSBuildSdkResolver.CachingWorkloadResolver.Resolve(String sdkReferenceName, IWorkloadManifestProvider manifestProvider, IWorkloadResolver workloadResolver)
   at Microsoft.NET.Sdk.WorkloadMSBuildSdkResolver.CachingWorkloadResolver.Resolve(String sdkReferenceName, String dotnetRootPath, String sdkVersion, String userProfileDir, String globalJsonPath)
   at Microsoft.NET.Sdk.WorkloadMSBuildSdkResolver.WorkloadSdkResolver.Resolve(SdkReference sdkReference, SdkResolverContext resolverContext, SdkResultFactory factory)
   at Microsoft.Build.BackEnd.SdkResolution.SdkResolverService.TryResolveSdkUsingSpecifiedResolvers(IReadOnlyList`1 resolvers, Int32 submissionId, SdkReference sdk, LoggingContext loggingContext, ElementLocation sdkReferenceLocation, String solutionPath, String projectPath, Boolean interactive, Boolean isRunningInVisualStudio, SdkResult& sdkResult, IEnumerable`1& errors, IEnumerable`1& warnings)""  C:\Program Files\dotnet\sdk\9.0.100-preview.7.[24](https://github.com/martincostello/costellobot/actions/runs/9972238641/job/27555071107?pr=1544#step:9:25)367.5\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.ImportWorkloads.props
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject[T1](IElementLocation elementLocation, String resourceName, T1 arg0)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImportsFromUnescapedImportExpressionConditioned(String directoryOfImportingFile, ProjectImportElement importElement, List`1& projects, SdkResult& sdkResult)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImports(String directoryOfImportingFile, ProjectImportElement importElement, SdkResult& sdkResult)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate()
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate(IEvaluatorData`4 data, Project project, ProjectRootElement root, ProjectLoadSettings loadSettings, Int[32](https://github.com/martincostello/costellobot/actions/runs/9972238641/job/27555071107?pr=1544#step:9:33) maxNodeCount, PropertyDictionary`1 environmentProperties, ILoggingService loggingService, IItemFactory`2 itemFactory, IToolsetProvider toolsetProvider, IDirectoryCacheFactory directoryCacheFactory, ProjectRootElementCacheBase projectRootElementCache, BuildEventContext buildEventContext, ISdkResolverService sdkResolverService, Int32 submissionId, EvaluationContext evaluationContext, Boolean interactive)
   at Microsoft.Build.Execution.ProjectInstance.Initialize(ProjectRootElement xml, IDictionary`2 globalProperties, String explicitToolsVersion, String explicitSubToolsetVersion, Int32 visualStudioVersionFromSolution, BuildParameters buildParameters, ILoggingService loggingService, BuildEventContext buildEventContext, ISdkResolverService sdkResolverService, Int32 submissionId, Nullable`1 projectLoadSettings, EvaluationContext evaluationContext, IDirectoryCacheFactory directoryCacheFactory)
   at Microsoft.Build.Execution.ProjectInstance..ctor(String projectFile, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, Nullable`1 projectLoadSettings, EvaluationContext evaluationContext, IDirectoryCacheFactory directoryCacheFactory, Boolean interactive)
   at Microsoft.Build.Execution.ProjectInstance..ctor(String projectFile, IDictionary`2 globalProperties, String toolsVersion)
   at Microsoft.DotNet.Workloads.Workload.Restore.WorkloadRestoreCommand.RunTargetToGetWorkloadIds(IEnumerable`1 allProjects)
   at Microsoft.DotNet.Workloads.Workload.Restore.WorkloadRestoreCommand.Execute()
   at System.CommandLine.Invocation.InvocationPipeline.Invoke(ParseResult parseResult)
   at System.CommandLine.ParseResult.Invoke()
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, TimeSpan startupTime, ITelemetry telemetryClient)

I can repro this as recently as 9.0.100-preview.7.24367.5:

Looking at the diff, the culprits could be either #41562 or #39991.

To Reproduce

  1. Clone https://github.com/martincostello/project-euler/commit/6727a6c588d99f750ca11248d51eecba67b46d27
  2. Run build.ps1 in the root of the repository
Forgind commented 1 month ago

This certainly looks bad!

I tried to reproduce it, and I'm having a little trouble. Any thoughts on what I'm doing wrong?

Windows --> windows powershell

I cloned your repo, checked out that commit, and verified that it included the global.json change to update the SDK.

I tried building, and it failed because of Join-Path. Apparently windows powershell is a bit out-of-date, and it doesn't permit Join-Path with more than 2 inputs, so I cobbled together several Join-Paths wherever you used one for > 2 arguments, and now it builds...

But it builds successfully with one warning that doesn't look relevant.

> ./build.ps1
The required version of the .NET SDK is not installed. Expected 9.0.100-preview.7.24367.5.
dotnet-install: Remote file https://dotnetbuilds.azureedge.net/public/Sdk/9.0.100-preview.7.24367.5/dotnet-sdk-9.0.100-preview.7.24367.5-win-x64.zip size is 265904062 bytes.
dotnet-install: Downloaded file https://dotnetbuilds.azureedge.net/public/Sdk/9.0.100-preview.7.24367.5/dotnet-sdk-9.0.100-preview.7.24367.5-win-x64.zip size is 265904062 bytes.
dotnet-install: The remote and local file sizes are equal.
dotnet-install: Extracting the archive.
dotnet-install: Binaries of dotnet can be found in C:\bugs\42211\project-euler\.dotnetcli\
dotnet-install: Note that the script does not resolve dependencies during installation.
dotnet-install: To check the list of dependencies, go to https://learn.microsoft.com/dotnet/core/install/windows#dependencies
dotnet-install: Installed version is 9.0.100-preview.7.24367.5
dotnet-install: Installation finished
Building solution...

Welcome to .NET 9.0!
---------------------
SDK Version: 9.0.100-preview.7.24367.5

Telemetry
---------
The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate, run 'dotnet dev-certs https --trust'
Learn about HTTPS: https://aka.ms/dotnet-https

----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
Restore complete (27.9s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  ProjectEuler succeeded (32.3s) → artifacts\bin\ProjectEuler\release\ProjectEuler.dll
  ProjectEuler.Benchmarks succeeded (6.1s) → artifacts\bin\ProjectEuler.Benchmarks\release\ProjectEuler.Benchmarks.dll
  ProjectEuler.Tests succeeded (5.1s) → artifacts\bin\ProjectEuler.Tests\release\ProjectEuler.Tests.dll

Build succeeded in 72.6s
Running tests...
Restore complete (1.2s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  ProjectEuler succeeded (0.2s) → artifacts\bin\ProjectEuler\release\ProjectEuler.dll
  ProjectEuler.Benchmarks succeeded (0.8s) → artifacts\bin\ProjectEuler.Benchmarks\release\ProjectEuler.Benchmarks.dll
  ProjectEuler.Tests succeeded (0.3s) → artifacts\bin\ProjectEuler.Tests\release\ProjectEuler.Tests.dll
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.0-preview.7.24365.1)
[xUnit.net 00:00:00.78]   Discovering: ProjectEuler.Tests
[xUnit.net 00:00:01.13]   Discovered:  ProjectEuler.Tests
[xUnit.net 00:00:01.14]   Starting:    ProjectEuler.Tests
The specified maximum value is invalid.
The specified maximum value is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified total is invalid.
The specified total is invalid.
At least 2 arguments are required.
At least 1 argument is required.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number of sides is invalid.
The specified number of sides is invalid.
The specified maximum value is invalid.
The specified maximum value is invalid.
The specified number of factors is invalid.
[xUnit.net 00:00:04.06]       Too slow.
The specified number of factors is invalid.
The specified number of divisors is invalid.
The specified maximum number is invalid.
The specified maximum number is invalid.
The specified maximum value is invalid.
The specified maximum value is invalid.
The specified number of sides is invalid.
The specified number of sides is invalid.
The specified number of sides is invalid.
The specified number of divisors is invalid.
The specified number of digits is invalid.
The specified number of digits is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
The specified number is invalid.
[xUnit.net 00:00:24.66]   Finished:    ProjectEuler.Tests
  ProjectEuler.Tests test succeeded with 1 warning(s) (29.2s)
    C:\bugs\42211\project-euler\.dotnetcli\sdk\9.0.100-preview.7.24367.5\Microsoft.TestPlatform.targets(46,5): warning : [xUnit.net 00:00:04.06]     Puzzle092_Returns_Correct_Solution [SKIP]

Test summary: total: 565, failed: 0, succeeded: 564, skipped: 1, duration: 29.1s
Build succeeded with 1 warning(s) in 33.1s

I suspect you have a workload set in an unexpected place, but I'd love to know more about it.

Here's my best guess: We've had a baseline workload set officially being produced for a while now, but it hasn't really had any effect, partially because it's unconditionally in the 8.0.100 folder, and we never looked there. dsplaisted's refactoring now permits looking across feature bands, and it found that workload set, but that workload set was never properly installed, which means it didn't come with all the manifests it was supposed to have, and that made this fail.

Mostly coincidentally, I think your current problem will be mitigated by https://github.com/dotnet/sdk/pull/41996 but only because that PR basically says "if you didn't ask for a workload version, we shouldn't give you a workload version," which means we wouldn't go looking for a manifest that was never installed.

Can you verify that you have a file named *.workloadset.json under a folder named 9.0.100-preview.7.24367.5 in your dotnet directory (under sdk-manifests\8.0.100\workloadsets, I think, but I haven't checked yet) and that it specifies the 8.0.1/8.0.100 version of microsoft.net.sdk.aspire? And can you try manually deleting that (along with parent folder(s) until they're not empty) and see if that resolves your issue?

martincostello commented 1 month ago

I typically just have the dailies ingest in the background in CI, rather than play around with them locally (unless I want to play with a brand new API that's just dropped). As these PRs are a failing, I haven't been pulling them down locally as I just pull from the target branch (dotnet-nightly).

Maybe it's something about the default SDK versions installed on the Windows GitHub Actions runners and actions/setup-dotnet works?

From the logs:

"C:\Program Files\PowerShell\7\pwsh.exe" -NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command & 'D:\a\_actions\actions\setup-dotnet\v4\externals\install-dotnet.ps1' -SkipNonVersionedFiles -Runtime dotnet -Channel LTS
dotnet-install: Remote file https://dotnetcli.azureedge.net/dotnet/Runtime/8.0.7/dotnet-runtime-8.0.7-win-x64.zip size is 33315858 bytes.
dotnet-install: Downloaded file https://dotnetcli.azureedge.net/dotnet/Runtime/8.0.7/dotnet-runtime-8.0.7-win-x64.zip size is 33315858 bytes.
dotnet-install: The remote and local file sizes are equal.
dotnet-install: Extracting the archive.
dotnet-install: Note that the script does not resolve dependencies during installation.
dotnet-install: To check the list of dependencies, go to https://learn.microsoft.com/dotnet/core/install/windows#dependencies
dotnet-install: Installed version is 8.0.7
dotnet-install: Installation finished
"C:\Program Files\PowerShell\7\pwsh.exe" -NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command & 'D:\a\_actions\actions\setup-dotnet\v4\externals\install-dotnet.ps1' -SkipNonVersionedFiles -Version 9.0.100-preview.7.24367.5
dotnet-install: Remote file https://dotnetbuilds.azureedge.net/public/Sdk/9.0.100-preview.7.24367.5/dotnet-sdk-9.0.100-preview.7.24367.5-win-x64.zip size is 265904062 bytes.
dotnet-install: Downloaded file https://dotnetbuilds.azureedge.net/public/Sdk/9.0.100-preview.7.24367.5/dotnet-sdk-9.0.100-preview.7.24367.5-win-x64.zip size is 265904062 bytes.
dotnet-install: The remote and local file sizes are equal.
dotnet-install: Extracting the archive.
dotnet-install: Note that the script does not resolve dependencies during installation.
dotnet-install: To check the list of dependencies, go to https://learn.microsoft.com/dotnet/core/install/windows#dependencies
dotnet-install: Installed version is 9.0.100-preview.7.24367.5
dotnet-install: Installation finished
Forgind commented 1 month ago

We're having issues reproducing this error. I mentioned what I did, and @baronfel looked into your github action and tried to reproduce it from there but also had difficulty.

It sounds like you haven't actually reproduced this locally. Can you try and see what comes of it? If it reproduces for you, we can try to see what's different between our setup and yours, and if it doesn't, we can look into what's different between CI and local builds.

Part of what's confusing is that the manifest that it references apparently should be installed already, so the fact that it can't find it means a poor error message, a broken install, or we're looking in the wrong spot. I was at least able to verify that the size of the SDK CI installed was the same as the size of the SDK that I installed, though it could've theoretically failed in extraction.

martincostello commented 1 month ago

I can't repro this locally either. If I checkout the branch and run build.ps1 it works and I can a file at the path .dotnetcli\sdk-manifests\9.0.100-preview.7\workloadsets\9.0.100-preview.7.24367.5\baseline.workloadset.json with the following content:

{
"Microsoft.NET.Sdk.Android": "34.99.0-preview.6.340/9.0.100-preview.6",
"Microsoft.NET.Sdk.iOS": "17.2.9714-net9-p6/9.0.100-preview.6",
"Microsoft.NET.Sdk.MacCatalyst": "17.2.9714-net9-p6/9.0.100-preview.6",
"Microsoft.NET.Sdk.macOS": "14.2.9714-net9-p6/9.0.100-preview.6",
"Microsoft.NET.Sdk.Maui": "9.0.0-preview.6.24327.7/9.0.100-preview.6",
"Microsoft.NET.Sdk.tvOS": "17.2.9714-net9-p6/9.0.100-preview.6",
"Microsoft.NET.Workload.Mono.ToolChain.Current": "9.0.0-preview.7.24365.1/9.0.100-preview.7",
"Microsoft.NET.Workload.Emscripten.Current": "9.0.0-preview.7.24352.2/9.0.100-preview.7",
"Microsoft.NET.Workload.Emscripten.net6": "9.0.0-preview.7.24352.2/9.0.100-preview.7",
"Microsoft.NET.Workload.Emscripten.net7": "9.0.0-preview.7.24352.2/9.0.100-preview.7",
"Microsoft.NET.Workload.Emscripten.net8": "9.0.0-preview.7.24352.2/9.0.100-preview.7",
"Microsoft.NET.Workload.Mono.ToolChain.net6": "9.0.0-preview.7.24365.1/9.0.100-preview.7",
"Microsoft.NET.Workload.Mono.ToolChain.net7": "9.0.0-preview.7.24365.1/9.0.100-preview.7",
"Microsoft.NET.Workload.Mono.ToolChain.net8": "9.0.0-preview.7.24365.1/9.0.100-preview.7",
"Microsoft.NET.Sdk.Aspire": "8.0.1/8.0.100",
}

Something that does stand out here (well, I had a squiggle when I opened the file in VS Code) regardless of whether it's actually related to this issue, I notice that the content isn't strictly valid JSON due to the trailing comma.

martincostello commented 1 month ago

I see the comma is catered for, so it's not that:

https://github.com/dotnet/sdk/blob/18985b172e4ed14fa4bb5c2056eae9c65bdac538/src/Resolvers/Microsoft.NET.Sdk.WorkloadManifestReader/WorkloadSet.cs#L72

martincostello commented 1 month ago

This is what's in the corresponding C:\Program Files\dotnet\sdk-manifests\8.0.100 folder on my local machine:

image

martincostello commented 1 month ago

Looks like the GitHub Actions runner comes with 8.0.0 pre-installed.

martincostello commented 1 month ago

I also noticed that between the last successful build with the prior working version and the latest, the GitHub Actions runner image has changed from 20240707.1.0 to 20240714.1.0 (changes).

I re-ran it to pick up the new runner image to see if that is the motivating factor, but it still passes.

martincostello commented 1 month ago

Messing about in this PR it seems that the GitHub Actions runner has aspire 8.0.0/8.0.100 installed on it initially:

Installed Workload Id      Manifest Version       Installation Source
---------------------------------------------------------------------
maui-windows               8.0.61/8.0.100         VS 17.10.35027.167 
maccatalyst                17.2.8053/8.0.100      VS 17.10.35027.167 
ios                        17.2.8053/8.0.100      VS 17.10.35027.167 
android                    34.0.95/8.0.100        VS 17.10.35027.167 
wasm-tools                 8.0.7/8.0.100          VS 17.10.35027.167 
aspire                     8.0.0/8.0.100          VS 17.10.35027.167 
Use `dotnet workload search` to find additional workloads to install.

Then if I run actions/setup-dotnet for both 8.0.100 and 9.0.100-preview.7.24367.22 then I end up with 8.0.2/8.0.100:

Installed Workload Id      Manifest Version       Installation Source            
---------------------------------------------------------------------------------
android                    34.0.113/8.0.100       SDK 8.0.300, VS 17.10.35027.167
aspire                     8.0.2/8.0.100          SDK 8.0.300, VS 17.10.35027.167
ios                        17.2.8078/8.0.100      SDK 8.0.300, VS 17.10.35027.167
maccatalyst                17.2.8078/8.0.100      SDK 8.0.300, VS 17.10.35027.167
maui-windows               8.0.61/8.0.100         SDK 8.0.300, VS 17.10.35027.167
wasm-tools                 8.0.7/8.0.100          SDK 8.0.300, VS 17.10.35027.167
Use `dotnet workload search` to find additional workloads to install.
martincostello commented 1 month ago

I also notice that the sorting there isn't stable - is it worth me opening a PR to add a sort to WorkloadListCommand.Execute()?

martincostello commented 1 month ago

I can't work out an obvious way to get 8.0.1/8.0.100 manually as a work around. It starts out with 8.0.0, but any command I seem to run related to workloads then wants to download 8.0.2 unless I add --skip-manifest-update.

MichalStrehovsky commented 1 month ago

I'm also running into this even with a super vanilla setup: https://github.com/MichalStrehovsky/bug42211/pulls

There's almost nothing in the repo, just a hello world console app and a hello world Github action setup.

martincostello commented 1 month ago

Thanks to https://github.com/dotnet/runtime/issues/69951#issuecomment-1140331799 I've been able to unblock myself from this issue by adding the following as the first step in my GitHub Actions workflow jobs:

# HACK Workaround for https://github.com/dotnet/sdk/issues/42211
- name: Update .NET Aspire workload
  if: runner.os == 'Windows'
  shell: pwsh
  run: |
    $manifest = @'
    {
      "microsoft.net.sdk.aspire": "8.0.1/8.0.100"
    }
    '@
    $manifestPath = Join-Path ${env:RUNNER_TEMP} "workload-manifest.json"
    $manifest | Out-File -FilePath $manifestPath -Encoding utf8
    dotnet workload update --advertising-manifests-only && `
    dotnet workload update --from-rollback-file $manifestPath

Obviously not ideal, but is better than being completely broken.

Forgind commented 1 month ago

Looks like the GitHub Actions runner comes with 8.0.0 pre-installed.

I guess I'm still confused as to why that should be so problematic. I would've thought it would have installed the 8.0.1 version along with the SDK. Perhaps there's a bug in the install script where it checks if that manifest (or perhaps a version of that manifest with the same major version?) is already installed and skips installing it if so? I wouldn't have expected that.

I also notice that the sorting there isn't stable - is it worth me opening a PR to add a sort to WorkloadListCommand.Execute()?

That sounds like a pretty reasonable change to me!

I can't work out an obvious way to get 8.0.1/8.0.100 manually as a work around. It starts out with 8.0.0, but any command I seem to run related to workloads then wants to download 8.0.2 unless I add --skip-manifest-update.

What commands did you try to run? In general, I think of there being two modes: "give me the latest" and "I want exactly this/these version(s)." You only get the latter behavior if you request to update/install --from-rollback-file, via global.json or --version (for workload set versions, not individual manifests), --from-history (still under review, so not yet), or in the install state (which you aren't supposed to be hand-editing. Any vanilla command will just grab the latest, which is 8.0.2, I guess.

martincostello commented 1 month ago

That sounds like a pretty reasonable change to me!

I'll open a PR tomorrow 👍

What commands did you try to run?

You can see the whole adventure here 😅: https://github.com/martincostello/api/pull/1841/commits

dotnet workload repair didn't work and trying to install older SDK didn't work either - everything seemed to basically start with 8.0.0 and then leapfrog up to 8.0.2.

martincostello commented 1 month ago

I'll open a PR tomorrow 👍

42247

baronfel commented 1 month ago

hey @martincostello - can you try setting the DOTNET_INSTALL_DIR env var in your build scripts to see if ensuring an isolated environment sidesteps the issue?

martincostello commented 1 month ago

Sure - I'll try that out tomorrow.

marcpopMSFT commented 1 month ago

hey @martincostello - can you try setting the DOTNET_INSTALL_DIR env var in your build scripts to see if ensuring an isolated environment sidesteps the issue?

To add on to what Chet said here, because the base image included an Admin-installed SDK and the install scripts installed a file-based install to the same location, our theory is that there is some incompatibility there. We're also exploring this config ourselves to try to confirm as we may need the install actions to default to an isolated environment in the future.

martincostello commented 1 month ago

Is this what you had in mind? If so, it didn't work.

martincostello commented 1 month ago

Looks like DOTNET_INSTALL_DIR also works around the issue (see https://github.com/martincostello/project-euler/pull/492).

Simplified workaround:

+- name: Set DOTNET_INSTALL_DIR
+  if: runner.os == 'Windows'
+  shell: pwsh
+  run: |
+    "DOTNET_INSTALL_DIR=${env:GITHUB_WORKSPACE}\.dotnet" >> ${env:GITHUB_ENV}

 - name: Setup .NET SDK
   uses: actions/setup-dotnet@v4
+  env:
+    DOTNET_INSTALL_DIR: ${{ env.DOTNET_INSTALL_DIR }}
Forgind commented 1 month ago

@martincostello,

Sorry I didn't get back to you too fast, but I put in a fix that I think will fix this. Can you check to see if (current main) works?

martincostello commented 1 month ago

Sure I can take a look - what was the change?

martincostello commented 1 month ago

Seems to be fixed (see martincostello/alexa-london-travel-site#2666), although there is a warning emitted during workload restore on Windows which I don't see using preview.7 nightlies where I'm using the workaround:

Warning: Workload garbage collection failed with error: Workload manifest microsoft.net.sdk.aspire: 8.1.0/8.0.100 from workload version 9.0.100-rc.1.24381.10 was not installed. Running "dotnet workload repair" may resolve this..

There's a different warning on Linux and macOS:

Warning: Workload garbage collection failed with error: Workload pack 'Microsoft.NET.Runtime.Emscripten.Node.net8' in manifest 'microsoft.net.workload.emscripten.net8' [/Users/runner/.dotnet/sdk-manifests/9.0.100-rc.1/microsoft.net.workload.emscripten.net8/9.0.0-rc.1.24373.3/WorkloadManifest.json] conflicts with manifest 'microsoft.net.workload.emscripten.current' [/Users/runner/.dotnet/sdk-manifests/8.0.100/microsoft.net.workload.emscripten.current/8.0.7/WorkloadManifest.json].
Forgind commented 1 month ago

https://github.com/dotnet/sdk/pull/42315 was the change that I thought would resolve the error you were seeing. It basically says that if you don't specifically request workload sets, don't treat it like your baseline workload set missing a manifest is a problem.

Interestingly, the symptom you're now seeing is very similar to another bug I'd fixed recently, but it has a different root cause. The baseline workload set was in the 8.0.100 folder (hardcoded), regardless of the SDK's real feature band. I fixed that, but I don't think that's the issue with this warning. The warning you're seeing is because the aspire manifest it expects really isn't installed. I suspect that if you were to run dotnet workload uninstall aspire, it would error.

Aspire still isn't installing properly, and I don't have a perfect answer on that still. We did see that it's trying to install the SDK into a directory that already has an SDK installed by VS with the 8.0.0 version of aspire, which is generally not good, but I don't know how to resolve that, unfortunately, and it's a bit outside my area of expertise.

martincostello commented 3 weeks ago

This seems to be resolved with the official .NET 9 preview.7 release, as well as in the rc.1 nightlies.