docker / buildx

Docker CLI plugin for extended build capabilities with BuildKit
Apache License 2.0
3.33k stars 448 forks source link

qemu segmentation fault installing pwsh modules #1422

Open DrBushyTop opened 1 year ago

DrBushyTop commented 1 year ago

Hi, I recently updated my M1 Max Macbook to Ventura, as well as the newest Docker Desktop version, and my previously building dockerfiles now fail on seemingly any Powershell module installation with either qemu: uncaught target signal 11 (Segmentation fault) - core dumped or target signal 6. It's been 1,5 months since I last built this specific image, so it's not necessarily related to either of the mentioned updates though.

Some specs: MacOS version: 13.0.1 (22A400) Docker Desktop Version: 4.14.1 (91661)

/Applications/Docker.app/Contents/MacOS/com.docker.diagnose check produces no fatal errors.

Building on ARM64 passes this fine.

Output from build:

❯ docker buildx build . -t "agent:nov23" --platform=linux/amd64
[+] Building 187.1s (7/7) FINISHED                                                                                                                                                      
 => [internal] load build definition from Dockerfile                                                                                                                               0.0s
 => => transferring dockerfile: 902B                                                                                                                                               0.0s
 => [internal] load .dockerignore                                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                    0.6s
 => [1/4] FROM docker.io/library/ubuntu:22.04@sha256:4b1d0c4a2d2aaf63b37111f34eb9fa89fa1bf53dd6e4ca954d47caebca4005c2                                                              0.0s
 => CACHED [2/4] RUN echo "APT::Get::Assume-Yes "true";" > /etc/apt/apt.conf.d/90assumeyes                                                                                         0.0s
 => [3/4] RUN apt-get update && apt-get install -y wget apt-transport-https software-properties-common   && wget -q https://packages.microsoft.com/config/ubuntu/22.04/packages  168.0s
 => ERROR [4/4] RUN pwsh -c "Install-Module Az -Force -SkipPublisherCheck -AllowClobber -Scope AllUsers"                                                                          18.4s 
------                                                                                                                                                                                  
 > [4/4] RUN pwsh -c "Install-Module Az -Force -SkipPublisherCheck -AllowClobber -Scope AllUsers":                                                                                      
#0 18.11 qemu: uncaught target signal 11 (Segmentation fault) - core dumped                                                                                                             
#0 18.43 Segmentation fault                                                                                                                                                             
------                                                                                                                                                                                  
ERROR: failed to solve: executor failed running [/bin/sh -c pwsh -c "Install-Module Az -Force -SkipPublisherCheck -AllowClobber -Scope AllUsers"]: exit code: 139

Here's the output for another module (Microsoft.Graph)

 => ERROR [10/18] RUN pwsh -c "./tools/modules.ps1"                                                                                                                               22.6s
------
 > [10/18] RUN pwsh -c "./tools/modules.ps1":
#0 22.15 Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
#0 22.15    at System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Collections.EnumerableExtensions.ReEnumerable[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.SafeAggregate[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Func`3<System.__Canon,System.__Canon,System.__Canon>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.JoinWith(System.Collections.Generic.IEnumerable`1<System.String>, System.String)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.JoinWithComma(System.Collections.Generic.IEnumerable`1<System.String>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicInterfaceExtensions.ToSignatureString(System.Reflection.MethodInfo)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicType+<>c.<Create>b__9_4(System.Collections.Generic.KeyValuePair`2<System.Reflection.MethodInfo,System.Reflection.MethodInfo>)
#0 22.15    at System.Linq.Enumerable+SelectListIterator`2[[System.Collections.Generic.KeyValuePair`2[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Collections.ReEnumerable`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ItemExists(Int32)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Collections.MutableEnumerable`1+Enumerator`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
#0 22.15    at System.Linq.Enumerable.Any[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.SafeAggregate[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Func`3<System.__Canon,System.__Canon,System.__Canon>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.JoinWith(System.Collections.Generic.IEnumerable`1<System.String>, System.String)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.JoinWithComma(System.Collections.Generic.IEnumerable`1<System.String>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicType+<>c__DisplayClass9_0.<Create>b__0(System.Type)
#0 22.15    at System.Linq.Enumerable+SelectEnumerableIterator`2[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Collections.ReEnumerable`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ItemExists(Int32)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Collections.MutableEnumerable`1+Enumerator`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
#0 22.15    at System.Linq.Enumerable.Aggregate[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Func`3<System.__Canon,System.__Canon,System.__Canon>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.SafeAggregate[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Func`3<System.__Canon,System.__Canon,System.__Canon>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Extensions.StringExtensions.JoinWith(System.Collections.Generic.IEnumerable`1<System.String>, System.String)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicType.Create(System.Type, Microsoft.PackageManagement.Internal.Utility.Collections.OrderedDictionary`2<System.Type,Microsoft.PackageManagement.Internal.Utility.Collections.List`2<System.Reflection.MethodInfo,System.Reflection.MethodInfo>>, Microsoft.PackageManagement.Internal.Utility.Collections.List`2<System.Delegate,System.Reflection.MethodInfo>, System.Collections.Generic.List`1<System.Reflection.MethodInfo>, Microsoft.PackageManagement.Internal.Utility.Collections.List`2<System.Type,System.Object>)
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicInterface.CreateProxy(System.Type, System.Object[])
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicInterface.DynamicCast(System.Type, System.Object[])
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicInterface.DynamicCast[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Object[])
#0 22.15    at Microsoft.PackageManagement.Internal.Utility.Plugin.DynamicInterfaceExtensions.As[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Object)
#0 22.15    at IPackageProvider_proxy_4.ResolvePackageSources(Microsoft.PackageManagement.Internal.Api.IRequest)
#0 22.15    at Microsoft.PackageManagement.Implementation.PackageProvider.<ResolvePackageSources>b__18_0(Microsoft.PackageManagement.Internal.Implementation.RequestObject)
#0 22.15    at Microsoft.PackageManagement.Internal.Implementation.RequestObject.<InvokeImpl>b__13_0()
#0 22.15    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
#0 22.15    at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
#0 22.16 qemu: uncaught target signal 6 (Aborted) - core dumped
#0 22.54 Aborted
------
ERROR: failed to solve: executor failed running [/bin/sh -c pwsh -c "./tools/modules.ps1"]: exit code: 134

Example Dockerfile

FROM ubuntu:22.04

# To make it easier for build and release pipelines to run apt-get,
# configure apt to not require confirmation (assume the -y argument by default)
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes

# Install powershell
# https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-linux
RUN apt-get update && apt-get install -y wget apt-transport-https software-properties-common \
  && wget -q https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb \
  && dpkg -i packages-microsoft-prod.deb \
  && apt-get update \
  && add-apt-repository universe \ 
  && apt-get install powershell \
  && rm -rf /var/lib/apt/lists/*

RUN pwsh -c "Install-Module Az -Force -SkipPublisherCheck -AllowClobber -Scope AllUsers"

Any ideas? Thanks!

crazy-max commented 1 year ago

Maybe similar to issues we have since QEMU v7, can you try with v6.2.0:

$ docker run --rm --privileged tonistiigi/binfmt --uninstall qemu-*
$ docker run --rm --privileged tonistiigi/binfmt:qemu-v6.2.0 --install all
DrBushyTop commented 1 year ago

Thanks for the good suggestion, but unfortunately this did not have any effect.

januszm commented 1 year ago

Not sure if this is 100% related but I get segmentation faults when on a Mac M1 Pro (arm64) I build a docker image for --platform=linux/amd64 project: a Ruby application that uses dependencies that require native extensions to be compiled in C, 9/10 cases end up with segmentation fault, but every now and then it works so this is clearly a bug in Docker/buildx/qemu

gcc: internal compiler error: Segmentation fault signal terminated program cc1
natescherer commented 11 months ago

I've confirmed that this is also a problem going in the opposite way: trying to build linux/arm64 on an amd64 system also causes the very same segfault.

UPDATE: I tested with QEMU 7.1.0, 7.0.0, 6.2.0, 6.1.0, 6.0.0, 5.2.0, all failed.

Hentioe commented 5 days ago

I build an image of linux/amd64 architecture on arm64 server, the source code comes from ImageMagick, and this error also occurs.

I want to know, do you have a solution? If the other way around, building linux/arm64/v8 on amd64 is successful.