search

docker / for-mac

Bug reports for Docker Desktop for Mac
https://www.docker.com/products/docker#/mac
2.43k stars 118 forks source link

Change to a symlink on a volume on the host side breaks the symlink within the container #7446

Open hvrauhal opened 1 week ago

hvrauhal commented 1 week ago

Description

We have a system that aims to do an atomic update of the complete contents in a directory by accessing the directory via a symlink and updating the symlink when the update should happen.

However, when modifying a symlink on a volume outside the container, the container sees an outdated symlink. This happens if the symlink is modified while the container is running and also when an entirely new container is started.

Reproduce

Run this script twice. The first run will work, but the second run will fail to list the contents behind the symlink initially and then recover once the upper directory is listed.

#!/bin/bash

set -eux

mkdir -p vol

cat > vol/test.sh << 'EOF'
#!/bin/sh

set -ux

counter=0

while true; do
  ls -il /vol/data/symlink

  counter=$((counter + 1))

  if [ $((counter % 5)) -eq 0 ]; then
    echo "ls the top dir"
    ls -il /vol/data/
  fi

  sleep 2
done;
EOF
chmod +x vol/test.sh

TARGET="vol/data"
mkdir -p "$TARGET"
NEW_CONTENTS=$RANDOM
RANDOM_TMP_SYMLINK=$RANDOM
mkdir "$TARGET"/"$NEW_CONTENTS"
echo "some contents" > "$TARGET"/"$NEW_CONTENTS/contentfile"
ln -s "$NEW_CONTENTS" "$TARGET"/"$RANDOM_TMP_SYMLINK"
mv -hf "$TARGET"/"$RANDOM_TMP_SYMLINK" "$TARGET"/symlink

cat << 'EOF'
Running a docker container to expose that the symlink manipulated from outside the container is broken
until the /vol/data/ is accessed either in the host or within the container
EOF

docker run --rm -v "$(pwd)"/vol:/vol debian:bookworm bash /vol/test.sh

The output of two runs of this script:

❯ ./host-test.sh
+ mkdir -p vol
+ cat
+ chmod +x vol/test.sh
+ TARGET=vol/data
+ mkdir -p vol/data
+ NEW_CONTENTS=14875
+ RANDOM_TMP_SYMLINK=4614
+ mkdir vol/data/14875
+ echo 'some contents'
+ ln -s 14875 vol/data/4614
+ mv -hf vol/data/4614 vol/data/symlink
+ cat
Running a docker container to expose that the symlink manipulated from outside the container is broken
until the /vol/data/ is accessed either in the host or within the container
++ pwd
+ docker run --rm -v /Users/rauhahe/f/ifc/packages/docker-testing/vol:/vol debian:bookworm bash /vol/test.sh
+ counter=0
+ true
+ ls -il /vol/data/symlink
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink -> 14875
+ counter=1
+ '[' 1 -eq 0 ']'
+ sleep 2
+ true
+ ls -il /vol/data/symlink
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink -> 14875
+ counter=2
+ '[' 2 -eq 0 ']'
+ sleep 2
^C+ true
+ ls -il /vol/data/symlink
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink -> 14875
+ counter=3
+ '[' 3 -eq 0 ']'
+ sleep 2
^C^C
got 3 SIGTERM/SIGINTs, forcefully exiting
❯ ./host-test.sh
+ mkdir -p vol
+ cat
+ chmod +x vol/test.sh
+ TARGET=vol/data
+ mkdir -p vol/data
+ NEW_CONTENTS=5731
+ RANDOM_TMP_SYMLINK=6926
+ mkdir vol/data/5731
+ echo 'some contents'
+ ln -s 5731 vol/data/6926
+ mv -hf vol/data/6926 vol/data/symlink
+ cat
Running a docker container to expose that the symlink manipulated from outside the container is broken
until the /vol/data/ is accessed either in the host or within the container
++ pwd
+ docker run --rm -v /Users/rauhahe/f/ifc/packages/docker-testing/vol:/vol debian:bookworm bash /vol/test.sh
+ counter=0
+ true
+ ls -il /vol/data/symlink
ls: cannot read symbolic link '/vol/data/symlink': Invalid argument
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink
+ counter=1
+ '[' 1 -eq 0 ']'
+ sleep 2
+ true
+ ls -il /vol/data/symlink
ls: cannot read symbolic link '/vol/data/symlink': Invalid argument
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink
+ counter=2
+ '[' 2 -eq 0 ']'
+ sleep 2
+ true
+ ls -il /vol/data/symlink
ls: cannot read symbolic link '/vol/data/symlink': Invalid argument
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink
+ counter=3
+ '[' 3 -eq 0 ']'
+ sleep 2
+ true
+ ls -il /vol/data/symlink
ls: cannot read symbolic link '/vol/data/symlink': Invalid argument
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink
+ counter=4
+ '[' 4 -eq 0 ']'
+ sleep 2
+ true
+ ls -il /vol/data/symlink
ls: cannot read symbolic link '/vol/data/symlink': Invalid argument
235 lrwxr-xr-x 1 root root 5 Oct  7 08:11 /vol/data/symlink
+ counter=5
+ '[' 0 -eq 0 ']'
+ echo 'ls the top dir'
+ ls -il /vol/data/
ls the top dir
total 0
233 drwxr-xr-x 3 root root 96 Oct  7 08:11 14875
236 drwxr-xr-x 3 root root 96 Oct  7 08:12 5731
238 lrwxr-xr-x 1 root root  4 Oct  7 08:12 symlink -> 5731
+ sleep 2
+ true
+ ls -il /vol/data/symlink
238 lrwxr-xr-x 1 root root 4 Oct  7 08:12 /vol/data/symlink -> 5731
+ counter=6
+ '[' 1 -eq 0 ']'
+ sleep 2
^C+ true
+ ls -il /vol/data/symlink
238 lrwxr-xr-x 1 root root 4 Oct  7 08:12 /vol/data/symlink -> 5731
+ counter=7
+ '[' 2 -eq 0 ']'
+ sleep 2
^C^C
got 3 SIGTERM/SIGINTs, forcefully exiting

Expected behavior

ls -il /vol/data/symlink

should always be able to list the contents within the symlinked directory.

docker version

Client:
 Version:           27.2.0
 API version:       1.47
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:14:45 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.34.2 (167172)
 Engine:
  Version:          27.2.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       3ab5c7d
  Built:            Tue Aug 27 14:15:41 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    27.2.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2-desktop.1
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2-desktop.2
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.13.0
    Path:     /Users/hvrauhal/.docker/cli-plugins/docker-scout

Server:
 Containers: 35
  Running: 33
  Paused: 0
  Stopped: 2
 Images: 63
 Server Version: 27.2.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.4-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 7.655GiB
 Name: docker-desktop
 ID: 8d82a9a0-a4f8-4763-b3ef-deb3a0c60417
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/hvrauhal/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Diagnostics ID

B8D5CF04-AD4D-4D98-B753-237BAB63AA98/20241007080524

Additional Info

No response

vukosid commented 1 day ago

@hvrauhal did you manage to resolve this?