search

moby / moby

The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
https://mobyproject.org/
Apache License 2.0
68.29k stars 18.59k forks source link

Unable to create macvlan network that overlap #46438

Open clementperon opened 11 months ago

clementperon commented 11 months ago

Description

Hi,

My host computer have several interfaces.

I would like to create multiple macvlan network with the same subnet for each interfaces.

I'm doing something technically wrong or is it a Docker limitation?

Reproduce

docker network create -d macvlan --subnet=192.168.1.1/24 -o parent=eno1 network-1 docker network create -d macvlan --subnet=192.168.1.1/24 -o parent=eno2 network-2 Error response from daemon: Pool overlaps with other one on this address space

Expected behavior

This should be acceptable

docker version

Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.2.0-26-generic
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.09GiB
 Name: hostname
 ID: 800cc3dc-3e9b-48d4-97b5-f4ffc0538240
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

akerouanton commented 10 months ago

Hi @clementperon,

Error response from daemon: Pool overlaps with other one on this address space

This come from the daemon's default IPAM driver. Once a subnet is reserved for a specific network, the driver will mark this subnet as unavailable and next time a network tries to use it, it will fail.

I would like to create multiple macvlan network with the same subnet for each interfaces.

Could you describe what you're trying to do? Also, how would you expect interfaces, routes, etc... to be configured?

clementperon commented 10 months ago

Hi @akerouanton,

Hi @clementperon,

Error response from daemon: Pool overlaps with other one on this address space

This come from the daemon's default IPAM driver. Once a subnet is reserved for a specific network, the driver will mark this subnet as unavailable and next time a network tries to use it, it will fail.

Yes, I understand that, but two subnet that belongs to two different interfaces should not interfer right?

I would like to create multiple macvlan network with the same subnet for each interfaces.

Could you describe what you're trying to do? Also, how would you expect interfaces, routes, etc... to be configured?

I'm spawing containers that are managing Devices Under Test. Each DUT is connected to a dedicated interface and have a default 192.168.1.201 IP after a factory reset.

To be able to have a replicable test, I configure my DUT to have the same Device IP and they communicate to the same Server IP. But each container have a dedicated interface.

CONTAINER1 (192.168.1.1/24) <--ENO1--> DUT1 (192.168.1.201/24) CONTAINER2 (192.168.1.1/24) <--ENO2--> DUT2 (192.168.1.201/24) CONTAINER3 (192.168.1.1/24) <--ENO3--> DUT3 (192.168.1.201/24)

Macvlan are associated to a different interface thus it should not be an issue to have the same IP range for different macvlan.

akerouanton commented 10 months ago

Macvlan are associated to a different interface thus it should not be an issue to have the same IP range for different macvlan.

Actually dockerd doesn't allow that because it'd be a source of connectivity issues if a container was connected to both networks.

Nonetheless, I thought it'd be possible to use the null IPAM driver to statically assign subnets and IP addresses with no validation whatsoever but it seems the macvlan / ipvlan drivers implicitly disallow its use. That's something we'd need to fix.

For now, unfortunately the workaround is to not use docker's networking features (ie. --network=host) and do it yourself.

clementperon commented 10 months ago

Macvlan are associated to a different interface thus it should not be an issue to have the same IP range for different macvlan.

Actually dockerd doesn't allow that because it'd be a source of connectivity issues if a container was connected to both networks.

Agree, but they aren't :). So the check is only looking at the IP_Address instead of the couple (IP_Address, Interface).

Nonetheless, I thought it'd be possible to use the null IPAM driver to statically assign subnets and IP addresses with no validation whatsoever but it seems the macvlan / ipvlan drivers implicitly disallow its use. That's something we'd need to fix.

I would be very happy If I could bypass the check an assign the IP address manually. I will test it to be sure it doesn't work.

For now, unfortunately the workaround is to not use docker's networking features (ie. --network=host) and do it yourself.

clementperon commented 10 months ago

@akerouanton thanks for your help unfortunately setting a static ip address without the subnet / ip_range give me the following error: failed to create network XXXX: Error response from daemon: ipv4 pool is empty