Assigned IP addresses not released when container is destroyed

kkuphal commented 5 years ago

Summary

Container created using docker run with simple network configuration

docker run -it -rm -p 162:162/udp --name snmptrapd --network vic-container-network snmptrapd

After container is terminated, docker inspect network shows IP addresses assigned to the deleted container that cannot be reused or removed

docker inspect vic-container-network [ { "Name": "vic-container-network", "Id": "369341720b6e041e4aabb48e6871677106dff0fea080cc542572b867b4a98256", "Created": "2019-04-23T18:21:44.649754577Z", "Scope": "", "Driver": "external", "EnableIPv6": false, "IPAM": { "Driver": "", "Options": {}, "Config": [ { "Subnet": "192.168.4.0/24", "IPRange": "192.168.4.140-192.168.4.199", "Gateway": "192.168.4.254" } ] }, "Internal": false, "Attachable": false, "Containers": { "0046e74521eb4b46ec6e0b382ed7e82da91980135356653d1567882d14554213": { "Name": "traefik-mcm995-103866763972", "EndpointID": "0046e74521eb4b46ec6e0b382ed7e82da91980135356653d1567882d14554213", "MacAddress": "", "IPv4Address": "192.168.4.140/24", "IPv6Address": "" }, "45fa4915e092b157dc37587e5d0aca466ff5397e010f9a28f3fade398f000446": { "Name": "snmptrapd", "EndpointID": "45fa4915e092b157dc37587e5d0aca466ff5397e010f9a28f3fade398f000446", "MacAddress": "", "IPv4Address": "192.168.4.142/24", "IPv6Address": "" }, "a4c3431b6a3ba2551759371ecc1061666d6977660c335b3e859f7825e8f45c60": { "Name": "snmptrapd", "EndpointID": "a4c3431b6a3ba2551759371ecc1061666d6977660c335b3e859f7825e8f45c60", "MacAddress": "", "IPv4Address": "192.168.4.143/24", "IPv6Address": "" }, "d03b544ec0f58f4e2bfd0187b30fa01507f7578f3356f74a0ca44f262e38396d": { "Name": "snmptrapd", "EndpointID": "d03b544ec0f58f4e2bfd0187b30fa01507f7578f3356f74a0ca44f262e38396d", "MacAddress": "", "IPv4Address": "192.168.4.141/24", "IPv6Address": "" } }, "Options": {}, "Labels": {} } ]

docker container rm d03 Error: No such container: d03

Environment information

vSphere and vCenter Server version

vCenter 6.5.0.23000 vSphere 6.5.0, 11925212

VIC Appliance version

vic-v1.5.2-7206-92ebfaf5

Configuration

Embedded or external PSC: Embedded
How was the OVA deployed? (Flex client, HTML5 client, ovftool): Flex client
Does the VIC appliance recieve configuration by DHCP? No. VIC is statically assigned
What stage of the Appliance Lifecycle is the VIC appliance in? Running
IP address of VIC appliance: 192.168.5.103
Hostname of VIC appliance: vic.fhpaschen.com
IP address of vCenter Server: 192.168.5.114
Hostname of vCenter Server: fhp-vcbe.fhpaschen.com

Details

Steps to reproduce

Create VCH with static IP range

--container-network vic-public:vic-container-network --container-network-ip-range vic-public:192.168.4.140-192.168.4.199 --container-network-gateway vic-public:192.168.4.254/24 --container-network-dns vic-public:192.168.4.98

Actual behavior

Allocated IP addresses are not released on container removal

Expected behavior

Allocated IP addresses are released on container removal

Logs

Troubleshooting attempted

[ ] Searched GitHub for existing issues. (Mention any similar issues under "See also", above.)
[ ] Searched the documentation for relevant troubleshooting guidance.
[ ] Searched for a relevant VMware KB article.

omnidepp commented 5 years ago

In addition, it seems like manually removing containers containers from a network via docker netwocker disconnectis not implemented yet. What other options are there besides destroying and recreating the entire network?

dafik commented 5 years ago

I'm affected this issue also.

This was working in 1.4.3

Now for workaround i must stop all containers, restart VCH , start containers. If i use docker-compose, every time i changed eg. env variable, containers are recreated, IP doesn't return to "free" pool, restart VCH is needed :/

yuyangbj commented 5 years ago

could you please run 'docker ps -a' to see the output? In 1.5.2 we fixed an ip conflict issue, so we do not release the ip address until the container is deleted. Terminated container is only stop the container. Please run docker rm to delete container.

yuyangbj commented 5 years ago

@kkuphal @dafik I am waiting for your response. I will close it if no comment.

dafik commented 5 years ago

$ cat docker-compose.yml 
version: "3.4"

networks:
  sinternal:
    internal: true
    ipam:
      config:
        - subnet: 172.69.0.0/16

services:
  db1:
    image: *********
    networks:
      sinternal:
        ipv4_address: 172.69.0.10

$ docker-compose ps
   Name                 Command              State    Ports  
-------------------------------------------------------------
test_db1_1   docker-entrypoint.sh postgres   Up      5432/tcp

lets change somthing in config eg. add env variable.

$ cat docker-compose.yml 
version: "3.4"

networks:
  sinternal:
    internal: true
    ipam:
      config:
        - subnet: 172.69.0.0/16

services:
  db1:
    image: *********
    networks:
      sinternal:
        ipv4_address: 172.69.0.10
    restart: always
    environment:
      POSTGRES_PASSWORD: xxxxx

restart services

$ docker-compose up -d
Recreating test_db1_1 ... 
Recreating test_db1_1 ... error

ERROR: for test_db1_1  Cannot start service db1: Server error from portlayer: Cannot reserve IP range 172.69.0.10 - 172.69.0.10.  Already in use

ERROR: for db1  Cannot start service db1: Server error from portlayer: Cannot reserve IP range 172.69.0.10 - 172.69.0.10.  Already in use

$ docker-compose ps   
         Name                        Command              State     Ports  
---------------------------------------------------------------------------
55a23d5e2054_test_db1_1   docker-entrypoint.sh postgres   Exit 0   5432/tcp
test_db1_1                docker-entrypoint.sh postgres   Exit 0   5432/tcp

dafik commented 5 years ago

@yuyangbj do you need any additional data to address this issue?

pmagn commented 4 years ago

We are also experiencing this issue on version 1.5.2 of VCH

samized commented 3 years ago

Me too, on version 1.5.5

bognerf commented 3 years ago

Same here, v1.5.6

This renders compose up -d to recreate changed containers with static IP addresses quite useless. As soon as one uses external networks to connect different docker-compose configurations interdependently, dynamic IP addresses are not useable.

Currently, one has to destroy a docker-compose container manually to release its IP address, before a recreation via docker-compose is even possible.

vmware / vic