search

apache / openwhisk

Apache OpenWhisk is an open source serverless cloud platform
https://openwhisk.apache.org/
Apache License 2.0
6.54k stars 1.17k forks source link

Last best hope in ansible is ephemeral couchdb, it works, but still with error messages #622

Closed houshengbo closed 8 years ago

houshengbo commented 8 years ago

When we run "ansible-playbook -i environments/local openwhisk.yml -e mode=clean" after the ansible deployment with ephemeral CouchDB, there are difficulties in removing registrators, consul, kafka, zookeeper, controller, etc, unpausing and removing other actions with the following errors raised:

ASK [consul : include] ********************************************************
Friday 10 June 2016  19:16:52 +0000 (0:00:02.646)       0:00:05.749 *********** 
skipping: [172.17.0.1]

TASK [consul : include] ********************************************************
Friday 10 June 2016  19:16:52 +0000 (0:00:00.038)       0:00:05.787 *********** 
included: /home/vagrant/openwhisk/ansible/roles/consul/tasks/clean.yml for 172.17.0.1

TASK [consul : remove registrator] *********************************************
Friday 10 June 2016  19:16:52 +0000 (0:00:00.063)       0:00:05.850 *********** 
fatal: [172.17.0.1]: FAILED! => {"changed": true, "failed": true, "msg": "Docker API Error: Unable to remove filesystem for 67972fc2c8757e290950e8371c67dd02b1dc71578013df3f02c5a2a3108cb577: remove /var/lib/docker/containers/67972fc2c8757e290950e8371c67dd02b1dc71578013df3f02c5a2a3108cb577/mqueue: device or resource busy"}
...ignoring

TASK [consul : remove consul] **************************************************
Friday 10 June 2016  19:16:52 +0000 (0:00:00.622)       0:00:06.473 *********** 
fatal: [172.17.0.1]: FAILED! => {"changed": true, "failed": true, "msg": "Docker API Error: Unable to remove filesystem for 65a0ec8569520a3c8cb6248eda8bae52036f2b1b39e9e24c1bc65e6aeb1f3003: remove /var/lib/docker/containers/65a0ec8569520a3c8cb6248eda8bae52036f2b1b39e9e24c1bc65e6aeb1f3003/mqueue: device or resource busy"}
...ignoring

TASK [consul : remove consul config directory] *********************************
Friday 10 June 2016  19:16:53 +0000 (0:00:00.634)       0:00:07.108 *********** 
changed: [172.17.0.1]

TASK [consul : remove consul log directory] ************************************
Friday 10 June 2016  19:16:53 +0000 (0:00:00.158)       0:00:07.266 *********** 
changed: [172.17.0.1]

TASK [consul : remove registrator log directory] *******************************
Friday 10 June 2016  19:16:53 +0000 (0:00:00.186)       0:00:07.452 *********** 
changed: [172.17.0.1]

PLAY [kafka] *******************************************************************

TASK [setup] *******************************************************************
Friday 10 June 2016  19:16:54 +0000 (0:00:00.185)       0:00:07.637 *********** 
ok: [172.17.0.1]

TASK [kafka : include] *********************************************************
Friday 10 June 2016  19:16:57 +0000 (0:00:03.832)       0:00:11.470 *********** 
skipping: [172.17.0.1]

TASK [kafka : include] *********************************************************
Friday 10 June 2016  19:16:58 +0000 (0:00:00.056)       0:00:11.526 *********** 
included: /home/vagrant/openwhisk/ansible/roles/kafka/tasks/clean.yml for 172.17.0.1

TASK [kafka : remove kafka] ****************************************************
Friday 10 June 2016  19:16:58 +0000 (0:00:00.056)       0:00:11.582 *********** 
fatal: [172.17.0.1]: FAILED! => {"changed": true, "failed": true, "msg": "Docker API Error: Unable to remove filesystem for 86af6a31fc2cc81682d52c7c1080d0609229fd389377ea7a9299aebd15bb9a0a: remove /var/lib/docker/containers/86af6a31fc2cc81682d52c7c1080d0609229fd389377ea7a9299aebd15bb9a0a/mqueue: device or resource busy"}
...ignoring

TASK [kafka : remove zookeeper] ************************************************
Friday 10 June 2016  19:17:08 +0000 (0:00:10.777)       0:00:22.360 *********** 
fatal: [172.17.0.1]: FAILED! => {"changed": true, "failed": true, "msg": "Docker API Error: Unable to remove filesystem for 2f5bf48c2cb5e34c0ed07101e681e464582ef9ec814d6850769c5b1d3bd15951: remove /var/lib/docker/containers/2f5bf48c2cb5e34c0ed07101e681e464582ef9ec814d6850769c5b1d3bd15951/mqueue: device or resource busy"}
...ignoring

PLAY [controllers] *************************************************************

TASK [setup] *******************************************************************
Friday 10 June 2016  19:17:09 +0000 (0:00:00.887)       0:00:23.248 *********** 
ok: [172.17.0.1]

TASK [controller : include] ****************************************************
Friday 10 June 2016  19:17:11 +0000 (0:00:01.908)       0:00:25.157 *********** 
skipping: [172.17.0.1]

TASK [controller : include] ****************************************************
Friday 10 June 2016  19:17:11 +0000 (0:00:00.030)       0:00:25.188 *********** 
included: /home/vagrant/openwhisk/ansible/roles/controller/tasks/clean.yml for 172.17.0.1

TASK [controller : remove controller] ******************************************
Friday 10 June 2016  19:17:11 +0000 (0:00:00.045)       0:00:25.233 *********** 
fatal: [172.17.0.1]: FAILED! => {"changed": true, "failed": true, "msg": "Docker API Error: Unable to remove filesystem for e8667d7c98e0c46d06e70db80178ee1f0a061a849d2a5f592333f3a2f6ef3e3b: remove /var/lib/docker/containers/e8667d7c98e0c46d06e70db80178ee1f0a061a849d2a5f592333f3a2f6ef3e3b/mqueue: device or resource busy"}
...ignoring

TASK [controller : remove controller log directory] ****************************
Friday 10 June 2016  19:17:22 +0000 (0:00:10.519)       0:00:35.753 *********** 
changed: [172.17.0.1]

PLAY [invokers] ****************************************************************

TASK [setup] *******************************************************************
Friday 10 June 2016  19:17:22 +0000 (0:00:00.133)       0:00:35.887 *********** 
ok: [172.17.0.1]

TASK [invoker : include] *******************************************************
Friday 10 June 2016  19:17:24 +0000 (0:00:01.870)       0:00:37.757 *********** 
skipping: [172.17.0.1]

TASK [invoker : include] *******************************************************
Friday 10 June 2016  19:17:24 +0000 (0:00:00.032)       0:00:37.790 *********** 
included: /home/vagrant/openwhisk/ansible/roles/invoker/tasks/clean.yml for 172.17.0.1

TASK [invoker : remove invoker] ************************************************
Friday 10 June 2016  19:17:24 +0000 (0:00:00.048)       0:00:37.839 *********** 
changed: [172.17.0.1]

TASK [invoker : unpause remaining actions] *************************************
Friday 10 June 2016  19:17:26 +0000 (0:00:01.853)       0:00:39.692 *********** 
fatal: [172.17.0.1]: FAILED! => {"changed": true, "cmd": "docker unpause $(docker ps -aq --filter name=wsk)", "delta": "0:00:00.097748", "end": "2016-06-10 19:17:26.425574", "failed": true, "rc": 1, "start": "2016-06-10 19:17:26.327826", "stderr": "docker: \"unpause\" requires a minimum of 1 argument.\nSee 'docker unpause --help'.\n\nUsage:\tdocker unpause [OPTIONS] CONTAINER [CONTAINER...]\n\nUnpause all processes within a container", "stdout": "", "stdout_lines": [], "warnings": []}
...ignoring

TASK [invoker : remove remaining actions] **************************************
Friday 10 June 2016  19:17:26 +0000 (0:00:00.257)       0:00:39.950 *********** 
fatal: [172.17.0.1]: FAILED! => {"changed": true, "cmd": "docker rm -f $(docker ps -aq --filter name=wsk)", "delta": "0:00:00.098052", "end": "2016-06-10 19:17:26.648984", "failed": true, "rc": 1, "start": "2016-06-10 19:17:26.550932", "stderr": "docker: \"rm\" requires a minimum of 1 argument.\nSee 'docker rm --help'.\n\nUsage:\tdocker rm [OPTIONS] CONTAINER [CONTAINER...]\n\nRemove one or more containers", "stdout": "", "stdout_lines": [], "warnings": []}
...ignoring
markusthoemmes commented 8 years ago

These are probably due to #199, as the container gets removed anyway, the error is ignored by ansible.

lehoanganh commented 8 years ago

correct, due to #199. this is a known problem, so i suggest to close this issue

lehoanganh commented 8 years ago

@houshengbo this is still a valid issue?

houshengbo commented 8 years ago

@lehoanganh let me verify it.

houshengbo commented 8 years ago

This bug still exists, but not a blocking issue. All the errors are marked as ignoring.

markusthoemmes commented 8 years ago

@houshengbo which bug specifically? The error messages you're seeing? If yes this is a dup of #199 as mentioned by @lehoanganh

rabbah commented 8 years ago

Closing as duplicate of #199.