dotnet / dotnet-docker

Docker images for .NET and the .NET Tools.
https://hub.docker.com/_/microsoft-dotnet
MIT License
4.49k stars 1.94k forks source link

Globalization issues when using dotnet/aspnet:9.0-azurelinux3.0-distroless-extra on ARM64 #6023

Closed craigktreasure closed 1 week ago

craigktreasure commented 2 weeks ago

Describe the bug

I have a simple Blazor application (using Blazorise in this case) that ran just fine on .NET 8 in either the 8.0-azurelinux3.0-distroless-extra or 8.0-cbl-mariner2.0-distroless-extra containers. After upgrading the application to .NET 9 RC 2 using the 9.0-azurelinux3.0-distroless-extra container image (Mariner 2.0 is not available for .NET 9), I began encountering globalization errors that I haven't been able to resolve. There appears to be something different between the globalization resources installed for 9.0-azurelinux3.0-distroless-extra compared to the others, which is unexpected when upgrading to .NET 9 or switching from another .NET 9 container image.

Update: After further investigation, this issue only appears when running the ARM64 variant of the image on an ARM64 device.

Which .NET image(s) are you using?

mcr.microsoft.com/dotnet/aspnet:9.0-azurelinux3.0-distroless-extra with sha256:e006bfc2276280f9f768c555a2870d3484b85a871ef9d3635eab29e9a5795c97

Steps to reproduce

See the minimal repro here and run on an ARM64 device.

Other information

The same code works in both .NET 8 and .NET 9 with the following container images (not exhaustive):

The same code does not work on .NET 9 with the following container images (not exhaustive):

The error encountered is:

System.Globalization.CultureNotFoundException: Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
cs is an invalid culture identifier.
   at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride)
   at Blazorise.Localization.TextLocalizerService.AddLanguageResource(String cultureName)
   at Blazorise.Localization.TextLocalizerService.ReadResource()
   at Blazorise.Localization.TextLocalizerService..ctor()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.AspNetCore.Components.ComponentFactory.<>c__DisplayClass9_0.<CreatePropertyInjector>g__Initialize|1(IServiceProvider serviceProvider, IComponent component)
   at Microsoft.AspNetCore.Components.ComponentFactory.InstantiateComponent(IServiceProvider serviceProvider, Type componentType, IComponentRenderMode callerSpecifiedRenderMode, Nullable`1 parentComponentId)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.InstantiateChildComponentOnFrame(RenderTreeFrame[] frames, Int32 frameIndex, Int32 parentComponentId)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewComponentFrame(DiffContext& diffContext, Int32 frameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewSubtree(DiffContext& diffContext, Int32 frameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.AppendDiffEntriesForRange(DiffContext& diffContext, Int32 oldStartIndex, Int32 oldEndIndexExcl, Int32 newStartIndex, Int32 newEndIndexExcl)
   at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.ComputeDiff(Renderer renderer, RenderBatchBuilder batchBuilder, Int32 componentId, ArrayRange`1 oldTree, ArrayRange`1 newTree)
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.RenderIntoBatch(RenderBatchBuilder batchBuilder, RenderFragment renderFragment, Exception& renderFragmentException)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()

I encountered this on:

Output of docker version

Client: Version: 27.3.1 API version: 1.47 Go version: go1.22.7 Git commit: ce12230 Built: Fri Sep 20 18:10:55 2024 OS/Arch: windows/arm64 Context: desktop-linux

Server: Docker Desktop 4.35.1 (173168) Engine: Version: 27.3.1 API version: 1.47 (minimum version 1.24) Go version: go1.22.7 Git commit: 41ca978 Built: Fri Sep 20 11:41:19 2024 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.7.21 GitCommit: 472731909fa34bd7bc9c087e4c27943f9835f111 runc: Version: 1.1.13 GitCommit: v1.1.13-0-g58aa920 docker-init: Version: 0.19.0 GitCommit: de40ad0

Output of docker info

Client: Version: 27.3.1 Context: desktop-linux Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.17.1-desktop.1 Path: C:\Program Files\Docker\cli-plugins\docker-buildx.exe compose: Docker Compose (Docker Inc.) Version: v2.29.7-desktop.1 Path: C:\Program Files\Docker\cli-plugins\docker-compose.exe debug: Get a shell into any image or container (Docker Inc.) Version: 0.0.37 Path: C:\Program Files\Docker\cli-plugins\docker-debug.exe desktop: Docker Desktop commands (Alpha) (Docker Inc.) Version: v0.0.15 Path: C:\Program Files\Docker\cli-plugins\docker-desktop.exe dev: Docker Dev Environments (Docker Inc.) Version: v0.1.2 Path: C:\Program Files\Docker\cli-plugins\docker-dev.exe extension: Manages Docker extensions (Docker Inc.) Version: v0.2.27 Path: C:\Program Files\Docker\cli-plugins\docker-extension.exe feedback: Provide feedback, right in your terminal! (Docker Inc.) Version: v1.0.5 Path: C:\Program Files\Docker\cli-plugins\docker-feedback.exe init: Creates Docker-related starter files for your project (Docker Inc.) Version: v1.3.0 Path: C:\Program Files\Docker\cli-plugins\docker-init.exe sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.) Version: 0.6.0 Path: C:\Program Files\Docker\cli-plugins\docker-sbom.exe scout: Docker Scout (Docker Inc.) Version: v1.14.0 Path: C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server: Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 12 Server Version: 27.3.1 Storage Driver: overlayfs driver-type: io.containerd.snapshotter.v1 Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 nvidia runc Default Runtime: runc Init Binary: docker-init containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111 runc version: v1.1.13-0-g58aa920 init version: de40ad0 Security Options: seccomp Profile: unconfined Kernel Version: 5.15.153.1-microsoft-standard-WSL2 Operating System: Docker Desktop OSType: linux Architecture: aarch64 CPUs: 12 Total Memory: 15.48GiB Name: docker-desktop ID: 906f5a08-c33f-41da-a52b-d8fa8d97bdfd Docker Root Dir: /var/lib/docker Debug Mode: false HTTP Proxy: http.docker.internal:3128 HTTPS Proxy: http.docker.internal:3128 No Proxy: hubproxy.docker.internal Labels: com.docker.desktop.address=npipe://\.\pipe\docker_cli Experimental: false Insecure Registries: hubproxy.docker.internal:5555 127.0.0.0/8 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support WARNING: No blkio throttle.write_bps_device support WARNING: No blkio throttle.read_iops_device support WARNING: No blkio throttle.write_iops_device support WARNING: daemon is not using the default seccomp profile

lbussell commented 2 weeks ago

Hi @craigktreasure, thank you for the detailed issue. I was not able to reproduce this issue using the instructions in the repo you provided.

I did have one suggestion that may help - I noticed that your Dockerfile relies on building the app locally and then copying binaries to the image. This can cause differences in the build from machine-to-machine.

I would recommend building the app inside the Dockerfile. You can check out our globalization sample for an example: https://github.com/dotnet/dotnet-docker/blob/0f8eee07e5c6fcc7243e10459a84b4b8090e506c/samples/globalapp/Dockerfile.azurelinux-distroless

I tested building and running the app with this Dockerfile as well and didn't run into any issues:

ARG sdkImageTag=9.0-azurelinux3.0
ARG baseImageTag=9.0-azurelinux3.0-distroless-extra

FROM mcr.microsoft.com/dotnet/sdk:${sdkImageTag} AS build
ARG TFM=net9.0
WORKDIR /source

# Copy project file and restore as distinct layers
COPY *.csproj .
RUN dotnet restore

# Copy source code and publish app
COPY . .
RUN dotnet publish --configuration Release --framework $TFM --no-restore -o /app

FROM mcr.microsoft.com/dotnet/aspnet:${baseImageTag}
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["./MyNewBlazoriseApp"]

You may also want to add a .dockerignore file so that you don't accidentally copy over the contents of any previous local builds:

**/bin
**/obj
**/out
craigktreasure commented 2 weeks ago

@lbussell I just tried it on an x64 machine and I didn't see it repro either. Do you happen to have an arm64 machine you can try it on? It might be specific to the arm64 image.

craigktreasure commented 2 weeks ago

@lbussell Updated my repro with your suggestions and retried on a few more machines. I can only reproduce it on ARM64 machines, so I believe it's specific to the ARM64 image produced. I'll update the title and description.

akoken commented 2 weeks ago

I got the same error on my machine.

OS: MacOS Sequoia 15.1 (Apple Silicon) Environment: Docker Desktop v4.35.1 Emulator: Rosetta

❯ docker run --rm -it -p 8080:8080 -e ASPNETCORE_ENVIRONMENT=Development mynewblazoriseapp:net9-azurelinux3
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/home/app/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning
warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
      No XML encryptor configured. Key {63bec15f-2796-416f-b922-12d5b5a08611} may be persisted to storage in unencrypted form.
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://[::]:8080
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /app
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
      Failed to determine the https port for redirect.
warn: Microsoft.AspNetCore.Components.Server.Circuits.RemoteRenderer[100]
      Unhandled exception rendering component: Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
      cs is an invalid culture identifier.
      System.Globalization.CultureNotFoundException: Only the invariant culture is supported in globalization-invariant mode. See https://aka.ms/GlobalizationInvariantMode for more information. (Parameter 'name')
      cs is an invalid culture identifier.
         at System.Globalization.CultureInfo..ctor(String name, Boolean useUserOverride)
         at Blazorise.Localization.TextLocalizerService.AddLanguageResource(String cultureName)
         at Blazorise.Localization.TextLocalizerService.ReadResource()
         at Blazorise.Localization.TextLocalizerService..ctor()
         at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
         at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
         at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
         at Microsoft.AspNetCore.Components.ComponentFactory.<>c__DisplayClass9_0.<CreatePropertyInjector>g__Initialize|1(IServiceProvider serviceProvider, IComponent component)
         at Microsoft.AspNetCore.Components.ComponentFactory.InstantiateComponent(IServiceProvider serviceProvider, Type componentType, IComponentRenderMode callerSpecifiedRenderMode, Nullable`1 parentComponentId)
         at Microsoft.AspNetCore.Components.RenderTree.Renderer.InstantiateChildComponentOnFrame(RenderTreeFrame[] frames, Int32 frameIndex, Int32 parentComponentId)
         at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewComponentFrame(DiffContext& diffContext, Int32 frameIndex)
         at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InitializeNewSubtree(DiffContext& diffContext, Int32 frameIndex)
         at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
         at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
         at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.InsertNewFrame(DiffContext& diffContext, Int32 newFrameIndex)
         at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.AppendDiffEntriesForRange(DiffContext& diffContext, Int32 oldStartIndex, Int32 oldEndIndexExcl, Int32 newStartIndex, Int32 newEndIndexExcl)
         at Microsoft.AspNetCore.Components.RenderTree.RenderTreeDiffBuilder.ComputeDiff(Renderer renderer, RenderBatchBuilder batchBuilder, Int32 componentId, ArrayRange`1 oldTree, ArrayRange`1 newTree)
         at Microsoft.AspNetCore.Components.Rendering.ComponentState.RenderIntoBatch(RenderBatchBuilder batchBuilder, RenderFragment renderFragment, Exception& renderFragmentException)
         at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
lbussell commented 2 weeks ago

Thanks for the additional context. We'll try to repro this specifically on ARM64.

mthalman commented 2 weeks ago

Found the problem. The .NET Dockerfile is misconfigured:

https://github.com/dotnet/dotnet-docker/blob/0f8eee07e5c6fcc7243e10459a84b4b8090e506c/src/aspnet/9.0/azurelinux3.0-distroless-extra/arm64v8/Dockerfile#L23

This should be based on the extra runtime image tag instead: 9.0.0-rc.2-azurelinux3.0-distroless-extra-arm64v8

Because it doesn't have the right base, it causes it to have DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true.

lbussell commented 2 weeks ago

@craigktreasure we've just published a fix in the nightly repo. Can you please try your scenario with the image: mcr.microsoft.com/dotnet/nightly/aspnet:9.0-azurelinux3.0-distroless-extra-arm64v8@sha256:47100c385f5865ef1422114e84a75ad3b7942fda05e92ea4bd057e748d0bdd40?

If it's all working, then we'll publish the fix in the main repo (non-nightly) next week.

craigktreasure commented 2 weeks ago

@lbussell, just tried it locally (removed the duplicate sha256) and it did work. Assuming the 9.0-azurelinux3.0-distroless-extra-arm64v8 and 9.0-azurelinux3.0-distroless-extra (for the ARM64 platform) are built the same, that should do it! Sadly, MCR doesn't show multi-platform image details, so it's not totally clear whether they're equivalent or not without looking at the build process

lbussell commented 2 weeks ago

Oops yes, that was a typo. They are built the same, just wanted to be as specific as possible.

PS> docker buildx imagetools inspect mcr.microsoft.com/dotnet/nightly/aspnet:9.0-azurelinux3.0-distroless-extra

Name:      mcr.microsoft.com/dotnet/nightly/aspnet:9.0-azurelinux3.0-distroless-extra
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:1b20007a9f3e6cc6ede5731ee533f768b591775cf004c1baa7a5c160e6abe7a4

Manifests: 
  Name:      mcr.microsoft.com/dotnet/nightly/aspnet:9.0-azurelinux3.0-distroless-extra@sha256:9525d423ae642db4ca17cb03d19d0a9261ed38756e5fc24720e635062080a792
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      mcr.microsoft.com/dotnet/nightly/aspnet:9.0-azurelinux3.0-distroless-extra@sha256:47100c385f5865ef1422114e84a75ad3b7942fda05e92ea4bd057e748d0bdd40
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64

I'm working on better tests for this scenario now. Thanks for the detailed bug report and for testing out our images early!

lbussell commented 1 week ago

These fixes were rolled out yesterday with .NET 9. Feel free to re-open if that is not the case. Thanks again for the report!