search

ansible-collections / community.docker

Community Docker Collection for Ansible: modules and plugins for working with Docker
https://galaxy.ansible.com/ui/repo/published/community/docker/
GNU General Public License v3.0
197 stars 110 forks source link

docker_network fails when trying to update network connected to swarm service #14

Open kaysond opened 4 years ago

kaysond commented 4 years ago

Migrated from https://github.com/ansible/ansible/issues/65742

SUMMARY

docker_network doesn't check if a network is connected to any services before trying to remove it

ISSUE TYPE
COMPONENT NAME

docker_network

ANSIBLE VERSION
ansible 2.10.0.dev0
  config file = None
  configured module search path = [u'/home/administrator/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/ansible/lib/ansible
  executable location = /opt/ansible/bin/ansible
  python version = 2.7.15+ (default, Oct  7 2019, 17:39:04) [GCC 7.4.0]
CONFIGURATION
OS / ENVIRONMENT

Running ansible locally on Ubuntu 18.04

STEPS TO REPRODUCE

Create: ~/docker-compose.yml:

---
version: "3.7"

services:
  test:
    image: jwilder/whoami
    ports:
      - 8000:8000
    networks:
      - testnet

networks:
  testnet:
    external: true

playbook.yml:


---
- hosts: all
  connection: local
  tasks:
  - name: "create new swarm"
    docker_swarm:
      state: present

  - name: "create network"
    docker_network:
      name: "testnet"
      scope: "swarm"
      state: present

  - name: "deploy service"
    docker_stack:
      state: present
      name: "test"
      compose:
        - ~/docker-compose.yml

  - name: "update network ipam_config"
    docker_network:
      name: "testnet"
      scope: "swarm"
      internal: yes
      ipam_config:
        - subnet: 10.1.0.0/16
          gateway: 10.1.0.1
          iprange: 10.1.1.0/24
      state: present
      force: yes

Run ansible-playbook -i "127.0.0.1," -vvv ~/playbook.yml

EXPECTED RESULTS

testnet network should be recreated with the updated ipam_config.

Ideally, Ansible would also re-connect services based on the connected and append arguments, just as it would containers (I haven't tested this, but am guessing that because it does not appear to check services at all, the module will not do this)

ACTUAL RESULTS

The task chokes because the module attempts to remove the network while it is still connected to the test_test service.

The issue here is that services need to be updated to remove the network. The Docker command-line for this would be: docker service update --network-rm testnet test_test. 


administrator@testserver:~$ ansible-playbook -i "127.0.0.1," -vvvv playbook.yml
ansible-playbook 2.10.0.dev0
  config file = None
  configured module search path = [u'/home/administrator/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/ansible/lib/ansible
  executable location = /opt/ansible/bin/ansible-playbook
  python version = 2.7.15+ (default, Oct  7 2019, 17:39:04) [GCC 7.4.0]
No config file found; using defaults
setting up inventory plugins
Set default localhost to 127.0.0.1
Parsed 127.0.0.1, inventory source with host_list plugin
Loading callback plugin default of type stdout, v2.0 from /opt/ansible/lib/ansible/plugins/callback/default.pyc

PLAYBOOK: playbook.yml *****************************************************************************************************************************************
Positional arguments: playbook.yml
become_method: sudo
inventory: (u'127.0.0.1,',)
forks: 5
tags: (u'all',)
verbosity: 4
connection: smart
timeout: 10
1 plays in playbook.yml

PLAY [all] *****************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************
task path: /home/administrator/playbook.yml:2
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: administrator
<127.0.0.1> EXEC /bin/sh -c 'echo ~administrator && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094690.21-202394398444608 `" && echo ansible-tmp-1576094690.21-202394398444608="` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094690.21-202394398444608 `" ) && sleep 0'
<127.0.0.1> Attempting python interpreter discovery
<127.0.0.1> EXEC /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'python2.6'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
Using module file /opt/ansible/lib/ansible/modules/system/setup.py
<127.0.0.1> PUT /home/administrator/.ansible/tmp/ansible-local-9291d3Y_9_/tmpbCDh_P TO /home/administrator/.ansible/tmp/ansible-tmp-1576094690.21-202394398444608/AnsiballZ_setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/administrator/.ansible/tmp/ansible-tmp-1576094690.21-202394398444608/ /home/administrator/.ansible/tmp/ansible-tmp-1576094690.21-202394398444608/AnsiballZ_setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/administrator/.ansible/tmp/ansible-tmp-1576094690.21-202394398444608/AnsiballZ_setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/administrator/.ansible/tmp/ansible-tmp-1576094690.21-202394398444608/ > /dev/null 2>&1 && sleep 0'
ok: [127.0.0.1]
META: ran handlers

TASK [create new swarm] ****************************************************************************************************************************************
task path: /home/administrator/playbook.yml:5
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: administrator
<127.0.0.1> EXEC /bin/sh -c 'echo ~administrator && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094691.99-165539105977793 `" && echo ansible-tmp-1576094691.99-165539105977793="` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094691.99-165539105977793 `" ) && sleep 0'
Using module file /opt/ansible/lib/ansible/modules/cloud/docker/docker_swarm.py
<127.0.0.1> PUT /home/administrator/.ansible/tmp/ansible-local-9291d3Y_9_/tmp8BBdSG TO /home/administrator/.ansible/tmp/ansible-tmp-1576094691.99-165539105977793/AnsiballZ_docker_swarm.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/administrator/.ansible/tmp/ansible-tmp-1576094691.99-165539105977793/ /home/administrator/.ansible/tmp/ansible-tmp-1576094691.99-165539105977793/AnsiballZ_docker_swarm.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/administrator/.ansible/tmp/ansible-tmp-1576094691.99-165539105977793/AnsiballZ_docker_swarm.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/administrator/.ansible/tmp/ansible-tmp-1576094691.99-165539105977793/ > /dev/null 2>&1 && sleep 0'
ok: [127.0.0.1] => {
    "actions": [
        "No modification"
    ],
    "changed": false,
    "invocation": {
        "module_args": {
            "advertise_addr": null,
            "api_version": "auto",
            "autolock_managers": null,
            "ca_cert": null,
            "ca_force_rotate": null,
            "client_cert": null,
            "client_key": null,
            "debug": false,
            "default_addr_pool": null,
            "dispatcher_heartbeat_period": null,
            "docker_host": "unix://var/run/docker.sock",
            "election_tick": null,
            "force": false,
            "heartbeat_tick": null,
            "join_token": null,
            "keep_old_snapshots": null,
            "labels": null,
            "listen_addr": "0.0.0.0:2377",
            "log_entries_for_slow_followers": null,
            "name": null,
            "node_cert_expiry": null,
            "node_id": null,
            "remote_addrs": null,
            "rotate_manager_token": false,
            "rotate_worker_token": false,
            "signing_ca_cert": null,
            "signing_ca_key": null,
            "snapshot_interval": null,
            "ssl_version": null,
            "state": "present",
            "subnet_size": null,
            "task_history_retention_limit": null,
            "timeout": 60,
            "tls": false,
            "tls_hostname": "localhost",
            "validate_certs": false
        }
    },
    "result": "",
    "swarm_facts": {
        "CreatedAt": "2019-12-11T19:50:02.428917546Z",
        "DataPathPort": 4789,
        "DefaultAddrPool": [
            "10.0.0.0/8"
        ],
        "ID": "izh2k607cm14wxigl50uhi7je",
        "JoinTokens": {
            "Manager": "SWMTKN-1-3m709tij7cdul1uzrkbzpc4fih5qdv5uabfc8c0ebdbee0trg3-26ba25oq0murwhrq7i3se9dtv",
            "Worker": "SWMTKN-1-3m709tij7cdul1uzrkbzpc4fih5qdv5uabfc8c0ebdbee0trg3-4lqo4128vvnjmctlohec7pwud"
        },
        "RootRotationInProgress": false,
        "Spec": {
            "CAConfig": {
                "NodeCertExpiry": 7776000000000000
            },
            "Dispatcher": {
                "HeartbeatPeriod": 5000000000
            },
            "EncryptionConfig": {
                "AutoLockManagers": false
            },
            "Labels": {},
            "Name": "default",
            "Orchestration": {
                "TaskHistoryRetentionLimit": 5
            },
            "Raft": {
                "ElectionTick": 10,
                "HeartbeatTick": 1,
                "KeepOldSnapshots": 0,
                "LogEntriesForSlowFollowers": 500,
                "SnapshotInterval": 10000
            },
            "TaskDefaults": {}
        },
        "SubnetSize": 24,
        "TLSInfo": {
            "CertIssuerPublicKey": "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAElJ9nXDXrPNOeQHRw22HRCy+Njh6p4xJN100NXny4z6/FMHORBiySGAMwA3uxFT4APG6F4OpwInNe/wt3Yl4wQw==",
            "CertIssuerSubject": "MBMxETAPBgNVBAMTCHN3YXJtLWNh",
            "TrustRoot": "-----BEGIN CERTIFICATE-----\nMIIBaTCCARCgAwIBAgIUGjRyDj4LrbBK8x9fUqWO+3eRVkYwCgYIKoZIzj0EAwIw\nEzERMA8GA1UEAxMIc3dhcm0tY2EwHhcNMTkxMjExMTk0NTAwWhcNMzkxMjA2MTk0\nNTAwWjATMREwDwYDVQQDEwhzd2FybS1jYTBZMBMGByqGSM49AgEGCCqGSM49AwEH\nA0IABJSfZ1w16zzTnkB0cNth0QsvjY4eqeMSTddNDV58uM+vxTBzkQYskhgDMAN7\nsRU+ADxuheDqcCJzXv8Ld2JeMEOjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMB\nAf8EBTADAQH/MB0GA1UdDgQWBBSDpOaG2zoLvCY+KtqaiT52VqClzjAKBggqhkjO\nPQQDAgNHADBEAiATVEjLDa81qyrD6p2hpMB6mlIJ8ARCRkXg+wNAZwNptQIgX1wJ\nf5u+cjlSEUkgWz2jQHchwB1i5WvczmBLMsJZYMY=\n-----END CERTIFICATE-----\n"
        },
        "UnlockKey": null,
        "UpdatedAt": "2019-12-11T19:50:02.552268297Z",
        "Version": {
            "Index": 10
        }
    }
}

TASK [create network] ******************************************************************************************************************************************
task path: /home/administrator/playbook.yml:9
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: administrator
<127.0.0.1> EXEC /bin/sh -c 'echo ~administrator && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094693.07-94116159261453 `" && echo ansible-tmp-1576094693.07-94116159261453="` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094693.07-94116159261453 `" ) && sleep 0'
Using module file /opt/ansible/lib/ansible/modules/cloud/docker/docker_network.py
<127.0.0.1> PUT /home/administrator/.ansible/tmp/ansible-local-9291d3Y_9_/tmpQ4cLzB TO /home/administrator/.ansible/tmp/ansible-tmp-1576094693.07-94116159261453/AnsiballZ_docker_network.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/administrator/.ansible/tmp/ansible-tmp-1576094693.07-94116159261453/ /home/administrator/.ansible/tmp/ansible-tmp-1576094693.07-94116159261453/AnsiballZ_docker_network.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/administrator/.ansible/tmp/ansible-tmp-1576094693.07-94116159261453/AnsiballZ_docker_network.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/administrator/.ansible/tmp/ansible-tmp-1576094693.07-94116159261453/ > /dev/null 2>&1 && sleep 0'
ok: [127.0.0.1] => {
    "ansible_facts": {
        "docker_network": {
            "Attachable": false,
            "ConfigFrom": {
                "Network": ""
            },
            "ConfigOnly": false,
            "Containers": {},
            "Created": "2019-12-11T19:56:35.685260011Z",
            "Driver": "bridge",
            "EnableIPv6": false,
            "IPAM": {
                "Config": [
                    {
                        "Gateway": "172.19.0.1",
                        "Subnet": "172.19.0.0/16"
                    }
                ],
                "Driver": "default",
                "Options": null
            },
            "Id": "ydi7ktuaxvyvf17xvs4kwcv6h",
            "Ingress": false,
            "Internal": false,
            "Labels": {},
            "Name": "testnet",
            "Options": {},
            "Scope": "swarm"
        }
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "api_version": "auto",
            "appends": false,
            "attachable": null,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "connected": [],
            "debug": false,
            "docker_host": "unix://var/run/docker.sock",
            "driver": "bridge",
            "driver_options": {},
            "enable_ipv6": null,
            "force": false,
            "internal": null,
            "ipam_config": null,
            "ipam_driver": null,
            "ipam_driver_options": null,
            "ipam_options": {
                "aux_addresses": null,
                "gateway": null,
                "iprange": null,
                "subnet": null
            },
            "labels": {},
            "name": "testnet",
            "scope": "swarm",
            "ssl_version": null,
            "state": "present",
            "timeout": 60,
            "tls": false,
            "tls_hostname": "localhost",
            "validate_certs": false
        }
    },
    "network": {
        "Attachable": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Created": "2019-12-11T19:56:35.685260011Z",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Config": [
                {
                    "Gateway": "172.19.0.1",
                    "Subnet": "172.19.0.0/16"
                }
            ],
            "Driver": "default",
            "Options": null
        },
        "Id": "ydi7ktuaxvyvf17xvs4kwcv6h",
        "Ingress": false,
        "Internal": false,
        "Labels": {},
        "Name": "testnet",
        "Options": {},
        "Scope": "swarm"
    }
}

TASK [deploy service] ******************************************************************************************************************************************
task path: /home/administrator/playbook.yml:15
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: administrator
<127.0.0.1> EXEC /bin/sh -c 'echo ~administrator && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094694.11-236202939741851 `" && echo ansible-tmp-1576094694.11-236202939741851="` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094694.11-236202939741851 `" ) && sleep 0'
Using module file /opt/ansible/lib/ansible/modules/cloud/docker/docker_stack.py
<127.0.0.1> PUT /home/administrator/.ansible/tmp/ansible-local-9291d3Y_9_/tmpPDRyQg TO /home/administrator/.ansible/tmp/ansible-tmp-1576094694.11-236202939741851/AnsiballZ_docker_stack.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/administrator/.ansible/tmp/ansible-tmp-1576094694.11-236202939741851/ /home/administrator/.ansible/tmp/ansible-tmp-1576094694.11-236202939741851/AnsiballZ_docker_stack.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/administrator/.ansible/tmp/ansible-tmp-1576094694.11-236202939741851/AnsiballZ_docker_stack.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/administrator/.ansible/tmp/ansible-tmp-1576094694.11-236202939741851/ > /dev/null 2>&1 && sleep 0'
ok: [127.0.0.1] => {
    "changed": false,
    "invocation": {
        "module_args": {
            "absent_retries": 0,
            "absent_retries_interval": 1,
            "compose": [
                "~/docker-compose.yml"
            ],
            "name": "test",
            "prune": false,
            "resolve_image": null,
            "state": "present",
            "with_registry_auth": false
        }
    },
    "rc": 0,
    "stderr": "",
    "stderr_lines": [],
    "stdout": "Updating service test_test (id: bdkua642raovsbpl97cpi240z)\n",
    "stdout_lines": [
        "Updating service test_test (id: bdkua642raovsbpl97cpi240z)"
    ]
}

TASK [update network ipam_config] ******************************************************************************************************************************
task path: /home/administrator/playbook.yml:22
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: administrator
<127.0.0.1> EXEC /bin/sh -c 'echo ~administrator && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094697.66-59738835547305 `" && echo ansible-tmp-1576094697.66-59738835547305="` echo /home/administrator/.ansible/tmp/ansible-tmp-1576094697.66-59738835547305 `" ) && sleep 0'
Using module file /opt/ansible/lib/ansible/modules/cloud/docker/docker_network.py
<127.0.0.1> PUT /home/administrator/.ansible/tmp/ansible-local-9291d3Y_9_/tmpti_GlQ TO /home/administrator/.ansible/tmp/ansible-tmp-1576094697.66-59738835547305/AnsiballZ_docker_network.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/administrator/.ansible/tmp/ansible-tmp-1576094697.66-59738835547305/ /home/administrator/.ansible/tmp/ansible-tmp-1576094697.66-59738835547305/AnsiballZ_docker_network.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/administrator/.ansible/tmp/ansible-tmp-1576094697.66-59738835547305/AnsiballZ_docker_network.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/administrator/.ansible/tmp/ansible-tmp-1576094697.66-59738835547305/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_docker_network_payload_iy71EX/ansible_docker_network_payload.zip/ansible/modules/cloud/docker/docker_network.py", line 692, in main
  File "/tmp/ansible_docker_network_payload_iy71EX/ansible_docker_network_payload.zip/ansible/modules/cloud/docker/docker_network.py", line 396, in __init__
  File "/tmp/ansible_docker_network_payload_iy71EX/ansible_docker_network_payload.zip/ansible/modules/cloud/docker/docker_network.py", line 615, in present
  File "/tmp/ansible_docker_network_payload_iy71EX/ansible_docker_network_payload.zip/ansible/modules/cloud/docker/docker_network.py", line 560, in remove_network
  File "/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/docker/api/network.py", line 187, in remove_network
    self._raise_for_status(res)
  File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
APIError: 400 Client Error: Bad Request ("rpc error: code = FailedPrecondition desc = network ydi7ktuaxvyvf17xvs4kwcv6h is in use by service bdkua642raovsbpl97cpi240z")

fatal: [127.0.0.1]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_version": "auto",
            "appends": false,
            "attachable": null,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "connected": [],
            "debug": false,
            "docker_host": "unix://var/run/docker.sock",
            "driver": "bridge",
            "driver_options": {},
            "enable_ipv6": null,
            "force": true,
            "internal": true,
            "ipam_config": [
                {
                    "aux_addresses": null,
                    "gateway": "10.1.0.1",
                    "iprange": "10.1.1.0/24",
                    "subnet": "10.1.0.0/16"
                }
            ],
            "ipam_driver": null,
            "ipam_driver_options": null,
            "ipam_options": {
                "aux_addresses": null,
                "gateway": null,
                "iprange": null,
                "subnet": null
            },
            "labels": {},
            "name": "testnet",
            "scope": "swarm",
            "ssl_version": null,
            "state": "present",
            "timeout": 60,
            "tls": false,
            "tls_hostname": "localhost",
            "validate_certs": false
        }
    },
    "msg": "An unexpected docker error occurred: 400 Client Error: Bad Request (\"rpc error: code = FailedPrecondition desc = network ydi7ktuaxvyvf17xvs4kwcv6h is in use by service bdkua642raovsbpl97cpi240z\")"
}

PLAY RECAP *****************************************************************************************************************************************************
127.0.0.1                  : ok=4    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
ansibullbot commented 4 years ago

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 4 years ago

cc @DBendit @WojciechowskiPiotr @akshay196 @chouseknecht @danihodovic @dariko @felixfontein @jwitko @kassiansun @keitwb @olsaki @tbouvet click here for bot help

felixfontein commented 4 years ago

For anyone interested in why this hasn't happened yet and what is necessary to implement this, see the discussion in ansible/ansible#65742.