"VMAllocationTracker.cpp:745 remove_shared_mem assertion failed error" under Rosetta on Apple Silicon Sonoma

rectalogic commented 7 months ago

Description

Running this amd64 container on macOS Sonoma 14.4 on an M2 mac under Rosetta gets this error:

assertion failed [rem_idx != -1]: Unable to find existing allocation for shared memory segment to unmap
(VMAllocationTracker.cpp:745 remove_shared_mem)
 Trace/breakpoint trap

Reproduce

> docker run --rm --init rectalogic/qtwebengine
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
assertion failed [rem_idx != -1]: Unable to find existing allocation for shared memory segment to unmap
(VMAllocationTracker.cpp:745 remove_shared_mem)
 Trace/breakpoint trap

This is the Dockerfile used to create the container:

# syntax = docker/dockerfile:1

# docker buildx build --platform linux/amd64 --load --tag rectalogic/qtwebengine .
# docker run --rm --init rectalogic/qtwebengine

# linux arm64 not supported by Qt https://doc.qt.io/qt-6/supported-platforms.html
FROM --platform=linux/amd64 ubuntu:jammy

ENV container docker
ENV DEBIAN_FRONTEND=noninteractive
ENV LANG=en_US.UTF-8
ENV LC_ALL=C.UTF-8
ENV GALLIUM_DRIVER=softpipe
ENV LIBGL_ALWAYS_SOFTWARE=1
ENV DRI_NO_MSAA=1

RUN apt-get -y update \
    && apt-get -y install \
    build-essential \
    cmake \
    curl \
    fontconfig \
    fonts-liberation \
    libasound2 \
    libglvnd-dev \
    libglx-mesa0 \
    libnss3 \
    libpulse-dev \
    libvulkan-dev \
    libx11-xcb1 \
    libxcb-cursor0 \
    libxcb-glx0 \
    libxcb-icccm4 \
    libxcb-image0 \
    libxcb-keysyms1 \
    libxcb-render-util0 \
    libxcb-shape0 \
    libxcb1 \
    libxcomposite1 \
    libxdamage1 \
    libxi6 \
    libxkbcommon-dev \
    libxkbcommon-x11-0 \
    libxrandr2 \
    libxrender1 \
    libxtst6 \
    mesa-vulkan-drivers \
    ninja-build \
    pkg-config \
    python3 \
    python3-pip \
    python3-venv\
    sudo \
    xvfb \
    xz-utils

RUN python3 -m venv venv && venv/bin/pip3 install aqtinstall \
    && venv/bin/python3 -m aqt install-qt linux desktop 6.6.2 --modules qtwebengine qtwebchannel qtpositioning --outputdir /usr/local/Qt

RUN useradd -ms /bin/bash -g root -G sudo qt
RUN echo "qt ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/qt && chmod 0440 /etc/sudoers.d/qt

COPY <<EOF /web.qml
  import QtWebEngine
  WebEngineView {}
EOF

USER qt
ENTRYPOINT ["/usr/bin/xvfb-run", "--server-args=-screen 0 640x480x24 -nolisten unix", "--error-file=/dev/stderr"]
CMD ["/usr/local/Qt/6.6.2/gcc_64/bin/qml", "/web.qml"]

Built with docker buildx build --platform linux/amd64 --load --tag rectalogic/qtwebengine .

Expected behavior

Should run without fatal errors. Running the container in docker on amd64 linux does not have this error.

docker version

Client:
 Cloud integration: v1.0.35+desktop.11
 Version:           25.0.3
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        4debf41
 Built:             Tue Feb  6 21:13:26 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.28.0 (139021)
 Engine:
  Version:          25.0.3
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       f417435
  Built:            Tue Feb  6 21:14:22 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    25.0.3
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1-desktop.4
    Path:     /Users/aw/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.6-desktop.1
    Path:     /Users/aw/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container. (Docker Inc.)
    Version:  0.0.24
    Path:     /Users/aw/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/aw/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.22
    Path:     /Users/aw/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/aw/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.0.1
    Path:     /Users/aw/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/aw/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.5.0
    Path:     /Users/aw/.docker/cli-plugins/docker-scout
WARNING: Plugin "/Users/aw/.docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /Users/aw/.docker/cli-plugins/docker-scan: no such file or directory

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 17
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.16-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 4.807GiB
 Name: docker-desktop
 ID: 6df6abf4-1b71-4342-b7bd-aac30b035329
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

ECBEBC37-A545-45B6-9DCA-FCE68405FE65/20240315183013

Additional Info

Other reports of a very similar issue, all on Sonoma Apple Silicon: https://forums.docker.com/t/unable-to-find-existing-allocation-for-shared-memory-segment-to-unmap/139868 https://forums.developer.apple.com/forums/thread/746618 https://bugs.openfoam.org/view.php?id=4060

Skazza94 commented 7 months ago

+1, I have the same exact error when running an amd64 image that is based on linuxserver/docker-baseimage-kasmvnc using Rosetta.

Looks like they changed something in Rosetta on macOS Sonoma 14.3 (see Apple Developer Forums).

It would be good to have some information from Docker devs about this problem.

thaJeztah commented 7 months ago

cc @dgageot @rumpl

dgageot commented 7 months ago

Hey everyone, sorry to hear that your workflow is broken... Indeed, things have been changed (broken?) in 14.3 and also in 14.4, that have an impact on this.

Have you tried running without --init? It seems to work for me.

I'll investigate some more and reach out to Apple to see if they can explain this behaviour.

dgageot commented 7 months ago

Interesting that this also fails with qemu (with --init only):

docker run --init --rm --platform=linux/amd64 rectalogic/qtwebengine
[72:72:0316/170314.189956:ERROR:sandbox_linux.cc(383)] InitializeSandbox() called with multiple threads in process renderer. Try waiting for /proc to be updated.
[72:72:0316/170315.298783:FATAL:thread_helpers.cc(104)] Current process is not mono-threaded! (iterations: 30)
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped

rectalogic commented 7 months ago

Yeah Apple definitely changed stuff recently https://blogs.oracle.com/java/post/java-on-macos-14-4

thaJeztah commented 7 months ago

Yeah Apple definitely changed stuff recently https://blogs.oracle.com/java/post/java-on-macos-14-4

Wow, interesting, that's a pretty big hammer, to change that to SIGKILL 😬;

With macOS 14.4, when a thread is operating in the write mode, if a memory access to a protected memory region is attempted, macOS will send the signal SIGKILL instead.

rectalogic commented 6 months ago

Have you tried running without --init? It seems to work for me.

Running without --init only appears to work because it does not actually run the crashing process at all. The Dockerfile uses xvfb-run which is a shell script that runs Xvfb then waits for SIGUSR1 before it runs the actual crashing process. Without --init it never gets the USR1 and never runs the process. This is probably related to https://github.com/docker/for-mac/issues/7122

somlyaip commented 6 months ago

I have the same error when trying to run a db2 container from an amd64 image using Rosetta on an M1 Pro.

kwalker-sugarcrm commented 6 months ago

FYI I had the same error for my greenplum image on an M3. I happened to have an old Docker.dmg that is version 4.13. I uninstalled 4.28, and installed the old 4.13. The error does not happen with version 4.13.

Dbini222 commented 6 months ago

has anyone been able to find a work around this issue? I'm also having the same problem

hifly81 commented 6 months ago

I have the same with 4.29

rectalogic commented 6 months ago

Here is a minimal testcase Dockerfile. It seems like detaching then reattaching the shmem segment is what triggers the bug:

# syntax = docker/dockerfile:1

FROM --platform=linux/amd64 ubuntu:jammy

RUN apt-get -y update && apt-get -y install build-essential

COPY <<EOF shmem.c
#include <stdio.h>
#include <sys/shm.h>
#include <fcntl.h>

int main ()
{
    int segment_id;
    char* shared_memory;

    segment_id = shmget(IPC_PRIVATE, 0x6400, IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR);

    shared_memory = (char*) shmat(segment_id, 0, 0);
    shmdt(shared_memory);

    shared_memory = (char*) shmat(segment_id, 0, 0);
    shmdt(shared_memory);

    return 0;
}
EOF

RUN <<EOF
    gcc -o shmem shmem.c -lrt
EOF

ENTRYPOINT ["./shmem"]

Build with docker buildx build --platform linux/amd64 --load --tag shmem . and run:

> docker run --rm -it shmem
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
assertion failed [rem_idx != -1]: Unable to find existing allocation for shared memory segment to unmap
(VMAllocationTracker.cpp:745 remove_shared_mem)

hifly81 commented 6 months ago

Hey guys, I finally resolved the issue by increasing the virtual disk limit by about one bar and it fixed the problem for me

I tried it but still same issue

cristian-cll commented 5 months ago

Same issue with rdesktop of linuxserver. I upgraded sonoma to 14.4.1 and I can no longer connect to this container via rdp. I have tried everything and nothing. It seems to be Rosetta's fault.

hotzenplotz5000 commented 5 months ago

Hello, I have the same problem here with a Qt-based application. I can reproduce the problem on macOS Sonoma 14.5 with Docker Desktop 4.29 (Rosetta activated) on a M1 and on a M3 Pro with the minimal example provided above. It would be great to have a workaround or fix for that. Anyone got any further already?

hhslepicka commented 5 months ago

I am having the same problem and looking for a workaround or, in an optimal scenario, a fix. Do you happen to have any news from anyone facing the same problem?

Shivareddy-Aluri commented 5 months ago

Hey guys i am facing the same issue as well. 14.4.1 macos sonoma, M2 docker 4.29.0

ench4 commented 5 months ago

I am having the same problem and looking for a workaround or, in an optimal scenario, a fix. Do you happen to have any news from anyone facing the same problem?

you can replace the files in the /Library/Apple/usr/libexec/oah/RosettaLinux folder with binaries from the macos version, which did not have this problem

don't forget to set execution rights :)

dgageot commented 5 months ago

Hi everyone! We're discussing with Apple so that they fix this issue. It is an issue with recent versions of Rosetta, shipped since 14.4.

robem commented 5 months ago

Just updated to Sonoma 14.5. The issue is still present. Verified with https://github.com/docker/for-mac/issues/7220#issuecomment-2050308136

ReturnRei commented 5 months ago

Same issue on Sonoma 14.5

mmartis commented 5 months ago

We are having the same issue with another Docker image that use xvfb-run and electron

superzeldalink commented 4 months ago

I am having the same problem and looking for a workaround or, in an optimal scenario, a fix. Do you happen to have any news from anyone facing the same problem?

you can replace the files in the /Library/Apple/usr/libexec/oah/RosettaLinux folder with binaries from the macos version, which did not have this problem

don't forget to set execution rights :)

I found a workaround based on this

Disable SIP
Download this (official Rosetta updater from , this is the version prior to the broken update)
Extract the pkg file: pkgutil --expand-full RosettaUpdateAuto.pkg RosettaUpdateAuto.pkg-expanded
Replace the files in RosettaUpdateAuto.pkg-expanded/RosettaUpdateAuto.pkg/Payload/Library/Apple/usr/libexec/oah/RosettaLinux to /Library/Apple/usr/libexec/oah/RosettaLinux (make sure you have created a backup)
Restart Docker Desktop

hologerry commented 4 months ago

I am having the same problem and looking for a workaround or, in an optimal scenario, a fix. Do you happen to have any news from anyone facing the same problem?

you can replace the files in the /Library/Apple/usr/libexec/oah/RosettaLinux folder with binaries from the macos version, which did not have this problem don't forget to set execution rights :)

I found a workaround based on this

Disable SIP

Download this (official Rosetta updater from , this is the version prior to the broken update)

Extract the pkg file: pkgutil --expand-full RosettaUpdateAuto.pkg RosettaUpdateAuto.pkg-expanded

Replace the files in RosettaUpdateAuto.pkg-expanded/RosettaUpdateAuto.pkg/Payload/Library/Apple/usr/libexec/oah/RosettaLinux to /Library/Apple/usr/libexec/oah/RosettaLinux (make sure you have created a backup)

Restart Docker Desktop

This workaround does not work for me. https://github.com/orbstack/orbstack/issues/1209

dgageot commented 4 months ago

Hi everyone! The issue is indeed still in 14.5. FWIW, apple is aware of it.

I can see workarounds mentioning the installation of an older version of Rosetta. While this does work, it'll break other things and it's also not supported nor encouraged by Apple. I also see mentions of orbstack having a different behaviour. I believe that their latest version also has the issue. Probably because it stopped using an older version of Rosetta as a workaround, for the reasons I mentioned.

I hate to say it but I believe the safest thing to do here is to wait for a fix, from Apple, to macOS/Rosetta itself.

joathan commented 4 months ago

Hello everyone, I managed to solve this problem by installing the Sonoma from scratch. I went into repair mode, formatted the drive and had it installed again.

mdcis-david commented 4 months ago

Hello everyone, I managed to solve this problem by installing the Sonoma from scratch. I went into repair mode, formatted the drive and had it installed again.

Just to confirm that the reinstall performed kept you on 14.5 and did not revert back to a prior version.

joathan commented 4 months ago

Hello everyone, I managed to solve this problem by installing the Sonoma from scratch. I went into repair mode, formatted the drive and had it installed again.

Just to confirm that the reinstall performed kept you on 14.5 and did not revert back to a prior version.

As far as I noticed, he got the latest version of the system.

Visão Geral do Software do Sistema:

  Versão do Sistema:    macOS 14.5 (23F79)
  Versão de Kernel: Darwin 23.5.0
  Volume de Inicialização:  Macintosh HD
  Modo Inicialização:   Normal
  Nome do Computador:   M1 Pro
  Nome de Usuário:  Joathan
  Memória Virtual Segura:   Ativada
  Proteção de Integridade do Sistema:   Ativada
  Tempo desde a inicialização:  2 dias, 20 horas e 33 minutos

claus-topholt-private commented 4 months ago

Any news on this?

jonathanreeves commented 3 months ago

For what it's worth, I'm seeing the same issue. MacOS 14.5, Macbook Pro M1 Max, Docker version 26.1.4, build 5650f9b. Would really love to find a viable workaround.

dgageot commented 3 months ago

Hi everyone, here's an internal build that you could try: Docker.dmg Does it fix your issues?

mmartis commented 3 months ago

Hi @dgageot, I installed the verison v4.32.0 (153994) and I'm still getting the same issue.

bschwert commented 3 months ago

This version 4.32.0 (153994) fixed it for me. I'm running the IBM ELM-Web-installer on debian:stable on MacOS 14.5 with Apple M3 Pro.

GiovanniLeo commented 3 months ago

Same issue here with MacBook Air M3 with Sonoma.

sinceronny commented 3 months ago

@rectalogic @dgageot could you please help to me understand why this is related to rossetta?

docker uses aarm64 containerd as CRI, and use QEMU to translate x86 based image to make it runnable on containerd. I don't get how ressetta got involved here.

rectalogic commented 3 months ago

@rectalogic @dgageot could you please help to me understand why this is related to rossetta?

Since 4.25 docker uses rosetta:

Docker now supports running x86-64 (Intel) binaries on Apple silicon with Rosetta 2

https://www.docker.com/blog/docker-desktop-4-25/

sinceronny commented 3 months ago

Thank you @rectalogic and @dgageot, do you know if Docker Desktop(aarm64) totally abandon QEMU and switch to Rossetta? or it will use QEMU in some situation?