MacVLAN. A network locks the gateway ip preventing to assign it

[brugnara]

I am excited about the features you have made in 1.12, so please let me say: thank you guys 😗

Now the lettuce... ;)

First of all, this is the network I have created

docker network rm vlan150
docker network create -d macvlan \
    --subnet=10.150.0.0/24 \
    --gateway=10.150.0.254 \
    -o macvlan_mode=bridge \
    -o parent=eth0.150 vlan150

Let me say here, the 10.150.0.254 does not exists in the network.

Now let's start a container on that network:

docker run --net=vlan150 --ip 10.150.0.254\
  -it --name server --rm alpine /bin/sh
docker: Error response from daemon: Address already in use.

This is not true. That address is not in use.. So let's circumnavigate that, this way:

docker run --net=vlan150 --cap-add NET_ADMIN \
  -it --name server --rm alpine /bin/sh
/ # ifconfig eth0 10.150.0.254 netmask 255.255.255.0

Now that container has the right gateway and every container connected to the vlan150 simply uses that as default gateway and everything work.

Am I missing something basic here? I saw pipeworks but I think it is too much for this simple use case.

Best regards.

[aboch]

@brugnara This is expected. You told the ipam driver to reserve the address 10.150.0.254 as you are using it for your network's gateway. Then you attempt to reserve the same address for your container. IPAM driver rejects the request becsase the address is already assigned.

[brugnara]

@aboch thank you for the quick reply. If I do not specify the gateway, the network driver will assign the 10.150.0.1 as default. How can I avoid the reservation as you mentioned?

[mavenugo]

@brugnara why would you not want docker to reserve the gateway IP ? Why are you reusing that IP in the container ? can you share the use-case ?

[aboch]

How can I avoid the reservation as you mentioned?

You can't avoid it. On network creation libnetwork will request the ipam driver for an address for the network's gateway, or will ask to reserve the one specified by the user during nw crewate.

IIUC, it looks like you are interested in having a container acting as the network gateway. Can you provide more info about the use case ? There could be other ways to achieve what you are trying to do.

[brugnara]

@mavenugo I need to set-up a squid as a transparent proxy for content injection. This must be setted as the default gw for many vpn clients. I will need to set some route for the returning back traffic because I need the original IP and I can not use NAT for this.

As I have told, everything is working if I manually change the ip for the squid machine as it becomes the gateway, because that IP is not assigned!!

Let me also say, in a MACVLAN it should be possible to assign the gateway IP because the gateway can be a container it self, like in this case.

If this is not a valid use case, please explain me what you think is wrong. Many thanks 👍

[blop]

+1 for allowing the gateway to be a container itself

[brugnara]

Any news?

[GordonTheTurtle]

@brugnara It has been detected that this issue has not received any activity in over 6 months. Can you please let us know if it is still relevant:

• For a bug: do you still experience the issue with the latest version?
• For a feature request: was your request appropriately answered in a later version?

Thank you! This issue will be automatically closed in 1 week unless it is commented on. For more information please refer to https://github.com/docker/libnetwork/issues/1926

[brugnara]

Due this issue I didn't used docker in our server farm. It is not relevant to me and not being relevant to you makes your choice easier.

On Mer 30 Ago 2017, 02:28 Gordon notifications@github.com wrote:

@brugnara https://github.com/brugnara It has been detected that this issue has not received any activity in over 6 months. Can you please let us know if it is still relevant:

• For a bug: do you still experience the issue with the latest version?
• For a feature request: was your request appropriately answered in a later version?

Thank you! This issue will be automatically closed in 1 week unless it is commented on. For more information please refer to #1926 https://github.com/docker/libnetwork/issues/1926

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/docker/libnetwork/issues/1447#issuecomment-325842413, or mute the thread https://github.com/notifications/unsubscribe-auth/ACpwBzJNHze61eCFNanouPwh-w6Vtor1ks5sdKy3gaJpZM4J-ARN .

[g00nix]

Is there any progress on this bug? This is really annoying. You can't use the first IP of the subnet even if you didn't set up a gateway.

[g00nix]

After going through the documentation and the code I noticed that this was done intentionally. If the user does not specify a gateway for a macvlan network, docker will enforce a gateway upon him. There is no escaping the wrath of the gateway.

As Docker is moving to bare metal, and IPv4 is reaching it's limits, people will continue to try to limit the number of public IPs and at the same time they will start using macvlan to get good performance and IP management. This means there will be cases where the default gateway for a macvlan network will be in another subnet than the subnet declare within the docker network. If you go even further and buy residential IPs instead of DC IPs, you will probably get a gateway that is not in the range, so that you can make full use of the residential subnet you pay for.

Programming docker to allow the gateway to be part of a different subnet might be harder (because it requires an extra route with the gateway/32 on eth0) than just allowing the user to actually get no gateway when he specifies no gateway. If you don't have a gateway that is managed by docker, you can just script it any way you want from bash. However, right now, one IP within the subnet is lost for the gateway.

To go around this issue, we are declaring a bigger subnet (so if the DC gives /27, we use /26 in docker), and just declare a dummy gateway that leads nowhere. After the container gets created, we delete the default route assigned by docker and just add a new one. This is clearly a hacky workaround that is lacking any sort of elegance.

Considering this case was opened 3 years ago with no progress, we might actually completely switch the world to IPv6 before we reach a resolution. That would also fix this mess with IPv4.

[jetndra]

Hi I m getting the same error with the below configuration

apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: xxx-oam-eth1 namespace: xxx spec: config: '{ "cniVersion": "0.3.0", "type": "macvlan", "master": "eth0", "mode": "bridge", "ipam": { "type": "host-local", "datastore": "kubernetes", "kubernetes": { "kubeconfig": "/etc/cni/net.d/whereabouts.d/whereabouts.kubeconfig" }, "subnet": "192.169.1.0/24", "rangeStart": "192.169.1.2", "rangeEnd": "192.169.1.240", "gateway": "192.169.1.1", "log_file" : "/tmp/whereabouts.log", "log_level" : "debug" } }

Normal Scheduled 19s default-scheduler Successfully assigned xxx/xxx-0 to k8s-worker-1 Warning FailedCreatePodSandBox 13s kubelet, k8s-worker-1 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bdc93dc123701afa226cd844fde27d3c9d20882a0f8c0bd117c4f1381d094df3": Multus: error adding pod to network "xxx-oam-eth1": delegateAdd: error invoking DelegateAdd - "macvlan": invalid CIDR : invalid CIDR address: Warning FailedCreatePodSandBox 6s kubelet, k8s-worker-1 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "4af83c0247d2fcda83cdf7a53990d84053a89f891839adc7f95a71c7490acec5": Multus: error adding pod to network "cms-oam-eth1": delegateAdd: error invoking DelegateAdd - "macvlan": invalid CIDR : invalid CIDR address: Can anyone plz help how to resolve this issue

[lo-w]

same issue here. the gateway IP is not used, we don't know why it cannot be assigned to the container. BR//lo

[hellmind]

same here, I want my router to be in a docker container .

[GMXTokyo]

Same issue here. Use-case: VyOS container acting as router.