Programs (PHP) using HUGEPAGES on Apple Silicon may fail with SIGILL (Illegal instruction)

Description

Disclaimer The following issue does not represent a very common configuration and the issue may be specific to PHP implementation. Docker for Mac does play a role here though, as the problem does not exist natively AND PHP specifically was mentioned in https://github.com/docker/for-mac/issues/7117#issuecomment-1880565915.

We have been experiencing a very uncomfortable bug for months on Apple Silicon (checked on M2 and newer) - PHP binary fails execution in approx. 1 out of 10 runs, when opcache.huge_code_pages is enabled. A very simple script can replicate the issue:

while true; do
    php -r 'return 0;'

    if [ $? -ne 0 ]; then
        break
    fi
done

The execution ends with SIGILL (Illegal instruction). I'm attaching a coredump output (via gdb) from such random execution.

For reproduction, we verified the issue exists with the official PHP docker distribution and also when compiled from source inside the target Debian container (according to https://php.watch/articles/compile-php-ubuntu).

It turns out that the culprit is in enabling opcache.huge_code_pages (https://www.php.net/manual/en/opcache.configuration.php#ini.opcache.huge_code_pages):

(minimum configuration for reproduction in opcache.ini)

opcache.enable=1
opcache.enable_cli=1
opcache.huge_code_pages=1

PHP version:

PHP 8.3.10 (cli) (built: Aug  2 2024 16:00:00) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.3.10, Copyright (c) Zend Technologies
    with Zend OPcache v8.3.10, Copyright (c), by Zend Technologies

gdb-core-dump-1.txt

Reproduce

docker run -it php bash (Inside the container)
docker-php-ext-install opcache
echo "opcache.enable=1" >> /usr/local/etc/php/conf.d/docker-php-ext-opcache.ini
echo "opcache.enable_cli=1" >> /usr/local/etc/php/conf.d/docker-php-ext-opcache.ini
echo "opcache.huge_code_pages=1" >> /usr/local/etc/php/conf.d/docker-php-ext-opcache.ini
while true; do php -r 'return 0;'; if [ $? -ne 0 ]; then break; fi; done (wait for SIGILL)

Expected behavior

Execution should not fail when huge pages are enabled.

docker version

Client:
 Version:           27.1.1
 API version:       1.46
 Go version:        go1.21.12
 Git commit:        6312585
 Built:             Tue Jul 23 19:54:12 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.33.0 (160616)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:14 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    27.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1-desktop.1
    Path:     /Users/x/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1-desktop.1
    Path:     /Users/x/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /Users/x/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.14
    Path:     /Users/x/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/x/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/x/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/x/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/x/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/x/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.11.0
    Path:     /Users/x/.docker/cli-plugins/docker-scout

Server:
 Containers: 18
  Running: 5
  Paused: 0
  Stopped: 13
 Images: 155
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.0-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 4.806GiB
 Name: docker-desktop
 ID: 2802ede3-1308-40d7-8f87-ade5b17d1864
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/x/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

4979D611-045D-44A8-B78A-89F8778E7332/20240824144450

Additional Info

No response

docker / for-mac