[BUG] `up` incorrectly recreates containers without changes + recreated containers have wrong files inside

rivit98 commented 2 months ago

Description

Hi, I have the following problem:

I have a repository where each service has its own separate directory containing a Dockerfile (same for each service) and a file named flag.txt (different for each service, the contents follow the service name). I noticed that after executing docker compose -p test up --build twice, some containers are recreated, but there was no configuration change. What's more - the flag.txt inside the recreated containers is completely different than expected (see the video).

Repository is available here: https://github.com/rivit98/docker-compose-experiments

https://github.com/user-attachments/assets/7a60e93b-ab9f-4f1f-a35f-a2f1d80522fe

Steps To Reproduce

git clone https://github.com/rivit98/docker-compose-experiments
cd docker-compose-experiments
docker compose -p test up --build
stop created containers with ctrl+c
docker compose -p test up --build - observe that some containers were recreated even if there were no changes in configuration/files/etc. If there were no recreated containers go to step 4.
find * -maxdepth 0 -type d | xargs -I{} bash -c "echo -n 'service: {} '; docker compose -p test exec -T {} cat /flag.txt; echo" - observe that containers that were recreated have wrong flag.txt file.

Compose Version

Docker Compose version v2.29.1

Docker Environment

Client: Docker Engine - Community
 Version:    27.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 14
  Running: 14
  Paused: 0
  Stopped: 0
 Images: 19
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-39-generic
 Operating System: Ubuntu 24.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 961.6MiB
 Name: localhost
 ID: 204f59ae-1ecc-436e-b7ab-d883414c3c73
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

idsulik commented 2 months ago

Hi! Couldn't reproduce it on main branch, on v2.29.1 tag and using version v2.28.1. always ok: 9, not_ok: 0

rivit98 commented 2 months ago

Hi @idsulik. I set up a new VM to redo the testing, and it turns out I can reproduce the issue on a clean Ubuntu 24.04 with the same Docker and Docker Compose versions as stated in the report.

jhrotko commented 3 days ago

@rivit98 could you try to reproduce with the latest version?

rivit98 commented 2 days ago

@jhrotko yes, I can reproduce it on fresh VM with the following docker and docker-compose versions:

root@localhost:~/docker-compose-experiments# cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04.1 LTS"

root@localhost:~/docker-compose-experiments# docker compose version
Docker Compose version v2.29.7

root@localhost:~/docker-compose-experiments# docker info
Client: Docker Engine - Community
 Version:    27.3.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.17.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.7
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 9
  Running: 9
  Paused: 0
  Stopped: 0
 Images: 13
 Server Version: 27.3.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-45-generic
 Operating System: Ubuntu 24.04.1 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 1.922GiB
 Name: localhost
 ID: c042fe84-c641-4d8d-887d-c6f00d7aa127
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

docker / compose