dotnet / sdk

Core functionality needed to create .NET Core projects, that is shared between Visual Studio and CLI
https://dot.net/core
MIT License
2.73k stars 1.07k forks source link

Runtime for migration bundle creation in Net7 #30466

Open DavidCMulder opened 1 year ago

DavidCMulder commented 1 year ago

When I create a migration bundle and do not specify a runtime, it looks like the subsequent build step does not specify a runtime. The creation of the bundle (from the verbose logging: dotnet publish --runtime win-x64) however does specify a runtime. This part then fails with 'error NETSDK1112: The runtime pack for Microsoft.NETCore.App.Runtime.win-x64 was not downloaded.'.

I can work around this by running a build specifying the runtime, and then a 'dotnet ef migrations bundle --no-build' succeeds.

So for example on my windows dev machine this fails:

dotnet ef migrations bundle -r linux-x64 -v

error: 'error NETSDK1112: The runtime pack for Microsoft.NETCore.App.Runtime.linux-x64 was not downloaded. Try running a NuGet restore with the RuntimeIdentifier 'linux-x64''

and this succeeds:

dotnet build -r linux-x64
dotnet ef migrations bundle -r linux-x64 --no-build

I think that when creating a bundle, the build and publishing of the bundle should specify the same runtime.

ajcvickers commented 1 year ago

This is a change in behavior in the SDK between 6 and 7. Moving to the sdk repo.

/cc @bricelam

nagilson commented 1 year ago

I assume /Bricelam owns the ef bundle creation. I'm guessing that you use a property like SelfContained or PublishAot, PublishReadyToRun, etc? These properties don't work unless a Runtime is specified so the SDK is giving one for you when publishing, but not for building because it would be unnecessary. Marking for team triage to confirm this is targeted to the right person, not sure who owns bundles.

I wouldn't be surprised if there isn't much that the ef migrations team could do about this, but IDK how their code works.

bricelam commented 1 year ago

Here's a repro project that takes dotnet ef out of the picture. The bundle project (that we call dotnet publish on) looks like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
    <PublishSingleFile>True</PublishSingleFile>
    <IncludeNativeLibrariesForSelfExtract>True</IncludeNativeLibrariesForSelfExtract>
    <ValidateExecutableReferencesMatchSelfContained>False</ValidateExecutableReferencesMatchSelfContained>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="7.0.2" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\App\App.csproj">
      <!-- HACK: Work around dotnet/sdk#10566 -->
      <GlobalPropertiesToRemove>SelfContained</GlobalPropertiesToRemove>
    </ProjectReference>
  </ItemGroup>

</Project>
nagilson commented 1 year ago

@bricelam FYI that hack shouldn't be needed anymore in net 7.0.

Thank you for creating a repro project. Are you expecting anything from the SDK on this end? I tried to use the repro project but publish.cmd passed successfully for me. Thanks.

bricelam commented 1 year ago

I wasn't able to repro it either. I think we're missing a piece of the puzzle.

FYI that hack shouldn't be needed anymore in net 7.0.

True, but trying to figure out which version of the SDK they're using is a lot more work than just leaving it in. 😉

nagilson commented 1 year ago

@DavidCMulder May you please provide a log (note the privacy notice) to help us debug this?

DavidCMulder commented 1 year ago

Hi @nagilson, I have attached the output in a zip, together with the powershell console output. BuildLogs.zip

DavidCMulder commented 1 year ago

I also had a go with the repro App project, and I have the same error there. dotnet ef migrations bundle -r linux-x64 -v fails To reproduce the error one needs to empty the local nuget cache (or at least delete the microsoft.aspnetcore.app.runtime.linux-x64 and microsoft.netcore.app.runtime.linux-x64 packages).

DavidCMulder commented 1 year ago

Or if I stick to the windows platform and remove the previously downloadeded microsoft.netcore.app.runtime.win-x64 package dotnet publish --runtime win10-x64 --no-self-contained on the bundle project fails:

PS C:\.....\efbundle> dotnet publish --runtime win10-x64 --no-self-contained
MSBuild version 17.5.0-preview-23061-01+040e2a90e for .NET
  Determining projects to restore...
  Restored C:\.....\efbundle\efbundle.csproj (in 270 ms).
  Restored C:\.....\App\App.csproj (in 270 ms).
C:\Program Files\dotnet\sdk\7.0.200\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(1142,5): warning NETSDK1179: One of '--self-contained' or '--no-self-contained' options are required when '--runtime' is used. [C:\.....\App\App.csproj]
C:\Program Files\dotnet\sdk\7.0.200\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(448,5): error NETSDK1112: The runtime pack for Microsoft.NETCore.App.Runtime.win-x64 was not downloaded. Try running a NuGe
t restore with the RuntimeIdentifier 'win-x64'. [C:\.....\App\App.csproj]
DavidCMulder commented 1 year ago

I also just came across a (I think) related issue: 29589

ultimaweapon commented 1 year ago

I just stumbled on this issue with GitHub Actions running on ubuntu-20.04. My project is .NET 6. Here are the error I got:

Warning: /usr/share/dotnet/sdk/7.0.202/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(1142,5): warning NETSDK1179: One of '--self-contained' or '--no-self-contained' options are required when '--runtime' is used. [/home/runner/work/erp/erp/src/Ultima.Erp/Ultima.Erp.csproj]
Error: /usr/share/dotnet/sdk/7.0.202/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(448,5): error NETSDK1112: The runtime pack for Microsoft.NETCore.App.Runtime.linux-x64 was not downloaded. Try running a NuGet restore with the RuntimeIdentifier 'linux-x64'. [/home/runner/work/erp/erp/src/Ultima.Erp/Ultima.Erp.csproj]
Error: /usr/share/dotnet/sdk/7.0.202/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(448,5): error NETSDK1112: The runtime pack for Microsoft.AspNetCore.App.Runtime.linux-x64 was not downloaded. Try running a NuGet restore with the RuntimeIdentifier 'linux-x64'. [/home/runner/work/erp/erp/src/Ultima.Erp/Ultima.Erp.csproj]

The steps that end up with the above error:

    - name: Build
      run: dotnet build -c Release src/Ultima.sln
    - name: Run tests
      run: dotnet test -c Release --no-build src/Ultima.sln
    - name: Export ERP
      run: dotnet publish -c Release -o dist/erp --no-build src/Ultima.Erp
    - name: Create migrations bundle
      run: dotnet ef migrations bundle -o update-db --configuration Release --no-build -v
      working-directory: src/Ultima.Erp
ultimaweapon commented 1 year ago

If someone getting hit the same issue as mine just use container options to specify 6.0 SDK like this:

jobs:
  build:
    name: Build
    runs-on: ubuntu-22.04
    container: mcr.microsoft.com/dotnet/sdk:6.0
cremor commented 1 year ago

This separate dotnet build -r <runtime> command is also required as a workaround if the dotnet ef migrations bundle command does not specify any runtime. It looks like the EF command implicitly uses the architecture of the build agent as the runtime of the migrations bundle. So for a Windows agent you have to use dotnet build --runtime win-x64.

The build log also contains this warning for the dotnet ef migrations bundle command, even though I didn't specifiy the runtime myself:

warning NETSDK1179: One of '--self-contained' or '--no-self-contained' options are required when '--runtime' is used.

But there is no --no-self-contained argument for the dotnet ef migrations bundle command!

nagilson commented 1 year ago

Sorry this issue got lost, thanks for providing the logs long ago. I think the best option is to do a dotnet restore -r RID with the runtimeIdentifier specified to fix an issue like this.

But there is no --no-self-contained argument for the dotnet ef migrations bundle command!

You should also be able to add <SelfContained>false</SelfContained to the project file instead. This error comes from the dotnet build/dotnet publish logic, which I'm guessing ef inherits from, and why it expects that flag to be there. A bit of an awkward user experience, for sure

cremor commented 1 year ago

It looks like the EF tools even uses the --no-self-contained argument, see first line in verbose build output below. But the warning is still triggered somehow.

Building bundle...
dotnet publish --runtime win10-x64 --output C:\Users\VssAdministrator\AppData\Local\Temp\rfp41dkr.bq2\publish --no-self-contained --configuration Release
MSBuild version 17.7.3+8ec440e68 for .NET
  Determining projects to restore...
  Restored C:\Users\VssAdministrator\AppData\Local\Temp\rfp41dkr.bq2\DbMigrationsBundle.csproj (in 1.1 sec).
  Restored D:\a\1\s\src\MyProject\MyProject.csproj (in 984 ms).
C:\hostedtoolcache\windows\dotnet\sdk\7.0.401\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(1156,5): warning NETSDK1179: One of '--self-contained' or '--no-self-contained' options are required when '--runtime' is used. [D:\a\1\s\src\MyProject\MyProject.csproj]
nagilson commented 1 year ago

FWIW, that error should be gone in .NET 8, but that is odd behavior. @bricelam Do the EF project targets perhaps undefine SelfContained somehow?

bricelam commented 1 year ago

Do the EF project targets perhaps undefine SelfContained somehow?

Yes. It used to cause errors if we let it flow to referenced projects.

https://github.com/dotnet/efcore/blob/7e4815d2b0a686cb51dc0d8c7d969eb0dcef36ac/src/ef/Generators/BundleProjectGenerator.tt#L21-L24

cremor commented 1 year ago

dotnet/sdk#10566 which is mentioned there seems to have been fixed for .NET 7. So maybe that workaround should be removed?

spawluk-zartis commented 1 year ago

flags -r and --runtime have different purposes.

-r configures target runtime --runtime configures runtime for NuGet restore

you should use --runtime win-x64

Usage: dotnet ef migrations bundle [options]

Options:
  -o|--output <FILE>                        The path of executable file to create.
  -f|--force                                Overwrite existing files.
  --self-contained                          Also bundle the .NET runtime so it doesn't need to be installed on the machine.
  -r|--target-runtime <RUNTIME_IDENTIFIER>  The target runtime to bundle for.
  -c|--context <DBCONTEXT>                  The DbContext to use.
  -p|--project <PROJECT>                    The project to use. Defaults to the current working directory.
  -s|--startup-project <PROJECT>            The startup project to use. Defaults to the current working directory.
  --framework <FRAMEWORK>                   The target framework. Defaults to the first one in the project.
  --configuration <CONFIGURATION>           The configuration to use.
  --runtime <RUNTIME_IDENTIFIER>            The runtime to use.
  --msbuildprojectextensionspath <PATH>     The MSBuild project extensions path. Defaults to "obj".
  --no-build                                Don't build the project. Intended to be used when the build is up-to-date.
  -h|--help                                 Show help information
  -v|--verbose                              Show verbose output.
  --no-color                                Don't colorize output.
  --prefix-output                           Prefix output with level.
cremor commented 1 year ago

But why do we even have to specify a runtime? It's not a required parameter for the dotnet ef migrations bundle command and it's also not mentioned in the docs.

baronfel commented 1 year ago

The ef tool shouldn't change the meaning of the well-known -r switch. Users expect the flag to mean --runtime like it does for every other dotnet CLI command.

cremor commented 5 months ago

Has this been fixed in .NET 8 (or EF Core 8)? Seems like I can't reproduce it any more with EF Core 8.

eerojaaskelainen commented 3 months ago

Still persist: Cannot generate bundle within ASP.NET Blazor project with WASM:

> dotnet ef migrations bundle
...
...
.nuget\packages\microsoft.net.sdk.webassembly.pack\8.0.7\build\Microsoft.NET.Sdk.WebAssembly.Browser.targets(231,5): error WASM0005: Unable to resolve WebAssembly runtime pack version
Microsoft.EntityFrameworkCore.Tools.CommandException: Build failed. Use --verbose to see errors.
   at Microsoft.EntityFrameworkCore.Tools.Commands.MigrationsBundleCommand.Execute(String[] args)
   at Microsoft.EntityFrameworkCore.Tools.Commands.CommandBase.<>c__DisplayClass0_0.<Configure>b__0(String[] args)
   at Microsoft.DotNet.Cli.CommandLine.CommandLineApplication.Execute(String[] args)
   at Microsoft.EntityFrameworkCore.Tools.Program.Main(String[] args)
Build failed. Use --verbose to see errors.
SauerkrautSundae commented 3 weeks ago

I upgrade my .NET 6 Blazor Project to a .NET8 Blazor Web Application with WASM. I use seperate EF Mirgraion projects for Oracle and for Postgres. Exact same error as @eerojaaskelainen . @nagilson, @bricelam is there a workaround so I can use dotnet ef migrations bundle?

This is part of my Log

dotnet ef migrations bundle --verbose --project ./Migrations/Project.Migrations.Postgres/Project.Migrations.Postgres.csproj --startup-project ./Server/Project.Server.csproj --context ProjectDbContext --output ./efbundle-postgres
...
Building bundle...
dotnet publish --runtime win-x64 --output C:\Users\<USERNAME>\AppData\Local\Temp\mxqvfvux.feo\publish --no-self-contained`
...
\.nuget\packages\microsoft.net.sdk.webassembly.pack\8.0.10\build\Microsoft.NET.Sdk.WebAssembly.Browser.targets(228,5): error WASM0005: Unable to resolve WebAssembly runtime pack version [C:\<PATH_TO_PROJECT>\ClientProject.csproj]
Microsoft.EntityFrameworkCore.Tools.CommandException: Build failed. Use --verbose to see errors.
   at Microsoft.EntityFrameworkCore.Tools.Commands.MigrationsBundleCommand.Execute(String[] args)
   at Microsoft.EntityFrameworkCore.Tools.Commands.CommandBase.<>c__DisplayClass0_0.<Configure>b__0(String[] args)
   at Microsoft.DotNet.Cli.CommandLine.CommandLineApplication.Execute(String[] args)
   at Microsoft.EntityFrameworkCore.Tools.Program.Main(String[] args)
Build failed. Use --verbose to see errors.