search

docker / compose

Define and run multi-container applications with Docker
https://docs.docker.com/compose/
Apache License 2.0
34.15k stars 5.25k forks source link

[BUG] On V2.24.0 a Circular reference Error #11343

Closed CodeZHXS closed 10 months ago

CodeZHXS commented 10 months ago

Description

When I use Docker Compose standalone v2.24.0, I encountered a bug. If you follow the steps below, you should be able to reproduce it:

  1. Pull some images: 
    docker pull yeasy/hyperledger-fabric-base:0.6-dp \
&& docker pull yeasy/hyperledger-fabric-peer:0.6-dp \
&& docker pull yeasy/hyperledger-fabric-membersrvc:0.6-dp \
&& docker pull yeasy/blockchain-explorer:latest \
&& docker tag yeasy/hyperledger-fabric-peer:0.6-dp hyperledger/fabric-peer \
&& docker tag yeasy/hyperledger-fabric-base:0.6-dp hyperledger/fabric-baseimage \
&& docker tag yeasy/hyperledger-fabric-membersrvc:0.6-dp hyperledger/fabric-membersrvc
  2. Create $2$ new YAML files，where 4-peers.yml depends on peer.yml.

Steps To Reproduce

When I use Docker Compose standalone v2.24.0, I encountered a bug. If you follow the steps below, you should be able to reproduce it:

  1. Pull some images: 
    docker pull yeasy/hyperledger-fabric-base:0.6-dp \
&& docker pull yeasy/hyperledger-fabric-peer:0.6-dp \
&& docker pull yeasy/hyperledger-fabric-membersrvc:0.6-dp \
&& docker pull yeasy/blockchain-explorer:latest \
&& docker tag yeasy/hyperledger-fabric-peer:0.6-dp hyperledger/fabric-peer \
&& docker tag yeasy/hyperledger-fabric-base:0.6-dp hyperledger/fabric-baseimage \
&& docker tag yeasy/hyperledger-fabric-membersrvc:0.6-dp hyperledger/fabric-membersrvc
  2. Create $2$ new YAML files, where 4-peers.yml depends on peer.yml. The peer.yml is: 
    
# This is the default base file to config env and command
# Notice that chaincode is executed inside docker in default net mode
# https://github.com/yeasy/docker-compose-files

Depends on the hyperledger/fabric-peer image.

This utilized the pbft as consensus

See https://github.com/hyperledger/fabric/blob/master/docs/dev-setup/devnet-setup.md#using-consensus-plugin for more details.

version: '2'

services: vp: # vp node no needing membersrvc image: yeasy/hyperledger-fabric-peer:0.6-dp

image: yeasy/hyperledger-fabric-peer:0.6-dp

image: hyperledger/fabric-peer:x86_64-0.6.1-preview

image: ibmblockchain/fabric-peer:x86_64-0.6.1-preview

restart: unless-stopped
environment:
  - CORE_PEER_ADDRESSAUTODETECT=true
  - CORE_PEER_NETWORKID=dev
  - CORE_LOGGING_LEVEL=info  #critical, error, warning, notice, info, debug
 #- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=fabric_pbft  # uncomment this to use specific network
  # The following section enables pbft consensus
  - CORE_PEER_VALIDATOR_CONSENSUS_PLUGIN=pbft  # noops, pbft
  - CORE_PBFT_GENERAL_MODE=batch  # batch. classic and sieve will be not supported again.
  - CORE_PBFT_GENERAL_N=4
  - CORE_PBFT_GENERAL_BATCHSIZE=2  # only useful when in batch mode
  - CORE_PBFT_GENERAL_TIMEOUT_REQUEST=2s
expose:
  - "7050"  # Rest
  - "7051"  # Grpc
  - "7052"  # Peer CLI
  - "7053"  # Peer Event
  - "7054"  # eCAP
  - "7055"  # eCAA
  - "7056"  # tCAP
  - "7057"  # eCAA
  - "7058"  # tlsCAP
  - "7059"  # tlsCAA
volumes: # docker.sock is mapped as the default CORE_VM_ENDPOINT
  - /var/run/docker.sock:/var/run/docker.sock
command: peer node start  # this repeats the default cmd to make sure

command: sh -c "sleep 5; peer node start" # peer node should have booting delay with CA

vp_sec: # vp node needing membersrvc suppot extends: service: vp environment:

version: '2'

services:

validating node as the root

vp0 will also be used for client interactive operations

If you want to run fabric command on the host, then map 7051:7051 to host

port, or use like CORE_PEER_ADDRESS=172.17.0.2:7051 to specify peer addr.

vp0: extends: file: peer.yml service: vp hostname: vp0 environment:

networks:

default:

external:

name: fabric_pbft

3. run ` docker-compose -f 4-peers.yml up`, then you will get error like:

Circular reference: vp0 in /root/pbft/4-peers.yml extends vp_sec in /root/pbft/peer.yml extends vp1 in /root/pbft/4-peers.yml extends vp_sec in /root/pbft/peer.yml

I have confirmed that there is no issue when using docker-compose v2.23.3.

### Compose Version

```Text
docker compose version
Docker Compose version v2.21.0

docker-compose version
Docker Compose version v2.24.0

Docker Environment

docker info
Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-91-generic
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.786GiB
 Name: linux2
 ID: 62466d6a-e2a2-4a89-871e-a6f96786af06
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://hub-mirror.c.163.com/
  https://docker.mirrors.ustc.edu.cn/
 Live Restore Enabled: false

root@linux2 ~/pbft
❯ systemctl daemon-reload
systemctl restart docker

root@linux2 ~/pbft 7s
❯ docker info
Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-91-generic
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.786GiB
 Name: linux2
 ID: 62466d6a-e2a2-4a89-871e-a6f96786af06
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

dkadioglu commented 10 months ago

The same happens for me, as I already described in the following issue at the Zabbix Docker repo: https://github.com/zabbix/zabbix-docker/issues/1121 With 2.23.3 the same compose files run without error.

ndeloof commented 10 months ago

I can reproduce this issue, will investigate this today and include a test-case with your example to ensure we don't get such a regression in the future

mauke commented 10 months ago

Just ran into this issue. Just for completeness, a reduced test case:

File a:

name: tmp

services:
  s0:
    extends:
      file: b
      service: s-base

  s1:
    extends:
      file: b
      service: s-base

File b:

services:
  s-base:
    image: busybox

  s-other:
    extends: s-base

Then run:

$ docker run -v ".:/mnt:ro" --rm docker:24.0.7-cli docker compose -f /mnt/a config
Circular reference:
  s1 in /mnt/a
  extends s-other in /mnt/b
  extends s0 in /mnt/a
  extends s-other in /mnt/b

("extends s-other" is of course wrong; the entry isn't actually used by any service.)