[BUG] Regression: array items[0,1] must be unique starting from 2.24.1

[paolomainardi]

Description

As per the subject, starting from 2.24.1, I am encountering this issue when there are overrides.

Steps To Reproduce

Create 2 files:

1. docker-compose.yaml

version: "3.8"

services:
  test:
    image: ubuntu:latest
    command: sleep infinity
    volumes:
      - ./src:/src

1. docker-compose.override.yaml

services:
  test:
    volumes:
      - ./src:/src

WIth 2.23.3:

❯ dc version
Docker Compose version 2.23.3
❯ dc down -v

With 2.24.1:

❯ ./dc-2.24.1 version
Docker Compose version v2.24.1
❯ ./dc-2.24.1 down -v
validating /home/paolo/temp/dc-compose/docker-compose.override.yml: services.test.volumes array items[0,1] must be unique

Compose Version

Docker Compose version v2.24.1

Docker Environment

❯ docker info
Client:
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.12.1
    Path:     /usr/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.23.3
    Path:     /usr/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 15
  Running: 9
  Paused: 0
  Stopped: 6
 Images: 141
 Server Version: 24.0.7
 Storage Driver: btrfs
  Btrfs:
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 71909c1814c544ac47ab91d2e8b84718e517bb99.m
 runc version:
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.11-2-lts
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 30.49GiB
 Name: paolo-cto-arch-wood
 ID: ZRJM:NTZC:JCYV:OSU3:VB2H:N2CW:ZCLD:PCGW:JGT5:B2BR:445A:GEHV
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[derekcentrico]

I don't mean to be a pain here, but I'm seeing similar on v2.25.0. Figured best to post on this thread versus a new issue as it seems related to my newb self?

"validating /home/docker/docker-compose.override.yml: services.scrutinyanalogj.devices array items[0,11] must be unique"

  scrutinyanalogj:
    ports:
      - '86:80'
      - '8886:8080'
    volumes:
      - '/home/docker/scrutinyanalogj:/opt/scrutiny/config'
      - '/home/docker/influxdb2:/opt/scrutiny/influxdb'
      - '/run/udev:/run/udev:ro'
    restart: always
    logging:
      options:
        max-size: 1g
    container_name: scrutinyanalogj
    environment:
      - PUID=1000
      - PGID=996
      - TZ=America/New_York
    devices:
      - '/dev/nvme0n1p1:/dev/nvme0'
      - '/dev/nvme1n1p1:/dev/nvme1'
      - '/dev/nvme2n1p1:/dev/nvme2'
      - '/dev/sda:/dev/sda'
      - '/dev/sdb:/dev/sdb'
      - '/dev/sdc:/dev/sdc'
      - '/dev/sdd:/dev/sdd'
      - '/dev/sde:/dev/sde'
      - '/dev/sdf:/dev/sdf'
      - '/dev/sdg:/dev/sdg'
      - '/dev/sdh:/dev/sdh'
    cap_add:
      - SYS_ADMIN
      - SYS_RAWIO
    image: ghcr.io/analogj/scrutiny:master-omnibus
    networks:
      vpnsys_net:
        ipv4_address: '172.22.0.109'

[ndeloof]

@derekcentrico tried to reproduce, works for me:

$ docker compose version
Docker Compose version v2.24.6-desktop.1
$ docker compose config
services:
  test:
    devices:
      - /dev/nvme0n1p1:/dev/nvme0
      - /dev/nvme1n1p1:/dev/nvme1
      - /dev/nvme2n1p1:/dev/nvme2
      - /dev/sda:/dev/sda
      - /dev/sdb:/dev/sdb
      - /dev/sdc:/dev/sdc
      - /dev/sdd:/dev/sdd
      - /dev/sde:/dev/sde
      - /dev/sdf:/dev/sdf
      - /dev/sdg:/dev/sdg
      - /dev/sdh:/dev/sdh

[derekcentrico]

@derekcentrico tried to reproduce, works for me:

$ docker compose version
Docker Compose version v2.24.6-desktop.1
$ docker compose config
services:
  test:
    devices:
      - /dev/nvme0n1p1:/dev/nvme0
      - /dev/nvme1n1p1:/dev/nvme1
      - /dev/nvme2n1p1:/dev/nvme2
      - /dev/sda:/dev/sda
      - /dev/sdb:/dev/sdb
      - /dev/sdc:/dev/sdc
      - /dev/sdd:/dev/sdd
      - /dev/sde:/dev/sde
      - /dev/sdf:/dev/sdf
      - /dev/sdg:/dev/sdg
      - /dev/sdh:/dev/sdh

Thanks for checking. You led me down the ultra newb path of a reboot. I don't understand how that mattered for anything as all was fine previously+ had just rebooted after an apt upgrade but hey it works now!

[alex-Symbroson]

I recently had the profiles array items[0,1] must be unique and updating to 4.28.0 fixed this error for me however, after that I got an error service "myservice" can't be used with 'extends' as it declare 'depends_on' so I extracted a base service definitions and defined multiple target services based on it, but after that the compose build command is trapped in an infinite resolve image config for docker.io/docker/dockerfile:1

Here is a minimal configuration where this happens:

version: '3'
services:
  service-base:
    build: .

  service-test:
    extends: 
      service: service-base
    build:
      target: .

[YtvwlD]

I'm getting this error also for security_opt. I'm on 2.26.1.

I have the following in both docker-compose.yaml and in docker-compose.override.yaml:

services:
   traefik:
     security_opt:
       - label:type:container_runtime_t

Is this expected or is this a bug? (This bug or a new one? :D)

[ndeloof]

@YtvwlD please open a new bug for this

[GuillaumeCisco]

I still experiment the issue with:

docker compose version
Docker Compose version v2.29.6

error message:

validating /home/$USER/Projects/docker-compose.yml: services.foo.extra_hosts array items[0,1] must be unique

Should have been resolved?

[ndeloof]

@GuillaumeCisco sounds like a separate, while comparable issue related to extra_hosts. Please fill a new issue so we can investigate

[GuillaumeCisco]

Thanks @ndeloof ,w ill do right away ;)