[BUG] Bind mounts no longer work in Bitbucket Pipelines in v2.29.3

[acarapetis]

Description

First of all, apologies for the framing here - I don't know enough about the Docker API to understand what the real problem is here, and calling this a bug is probably wrong.

Since upgrading our pipelines runner image to use compose v2.29.3, attempting to run a compose configuration that uses bind mounts in Bitbucket Pipelines is producing this error:

Error response from daemon: authorization denied by plugin pipelines: --mounts is not allowed

I'm guessing the relevant change here is #12078, and the problem is that compose has switched to using a new API for mounts that Bitbucket does not support.

Is there a way to revert to using the old API for bind mounts? Or do we just need to pin v2.29.2 for the foreseeable future?

Steps To Reproduce

services:
    blah:
        image: alpine
        command: ["echo", "hi"]
        volumes:
            - .:/dot

pipelines:
  default:
    - step:
      name: Test compose mount
      services:
        - docker
      script:
        - docker info
        - docker compose run blah

Push to a Bitbucket Cloud repository.

Compose Version

Docker Compose version v2.29.3

Docker Environment

+ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., v2.29.3)
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 25.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Authorization: pipelines
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7c3aca7a610df76212171d200ca3811ff6096eb8
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  userns
 Kernel Version: 5.15.0-1068-aws
 Operating System: Alpine Linux v3.19 (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 30.89GiB
 Name: 7d637b9f-a5ae-4e16-8a1f-c3f5e524f487-h8n8b
 ID: 1551bc4e-86a3-4330-8c96-4171cf1c4220
 Docker Root Dir: /var/lib/docker/165536.165536
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  10.2.215.48:5000
  127.0.0.0/8
 Registry Mirrors:
  http://10.2.215.48:5000/
 Live Restore Enabled: false
 Product License: Community Engine

Anything else?

No response

[phuongl-ampere]

I meet the same issue I have no /var/lib/mysql before.

My with docker-compose.yml:

services:
  db:
    image: mysql
    volumes:
      - /var/lib/mysql:/var/lib/mysql

(.venv) [root@phuongl 1.9.6]# docker-compose -f docker-compose.yml up
[+] Running 11/11
✔ db Pulled
[+] Running 0/0
[+] Running 1/1default  Creating
✔ Network 196_default  Created
⠋ Container 196-db-1   Creating
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /var/lib/mysql

but with v2.29.2 directory "/var/lib/mysql" will auto create.

[ndeloof]

mount API exists in docker engine for a while, I wonder it is not supported by bitbucket. The bind API is legacy which comes with many limitations. We rely on mount API as compose file syntax offers fine grained support of bind mounts which is only available with this newer API.

@phuongl-ampere can you please open a dedicated issue ?

[sbe-arg]

We had to pin the image in the runner to docker:27.1.2 wich runs the older compose.

Until bitbucket sorts their end in 2027 (or circa) hahahaha

[glours]

Can you test with the latest v2.29.7 release please?

[acarapetis]

Confirmed fixed in v2.29.7 :)