search

ccollicutt / swiftacular

Deploy OpenStack Swift with Ansible and Vagrant
BSD 2-Clause "Simplified" License
47 stars 13 forks source link

Clean run fails on ubuntu-cloud-keyring.python-pycurl #26

Closed brainstorm closed 10 years ago

brainstorm commented 10 years ago

After a fresh checkout and following the README.md steps, I'm getting this:

TASK: [common | install required packages to add havana repository] ***********
failed: [swift-storage-02] => (item=ubuntu-cloud-keyring,python-pycurl) => {"failed": true, "item": "ubuntu-cloud-keyring,python-pycurl"}
msg: No package matching 'ubuntu-cloud-keyring' is available
failed: [swift-storage-03] => (item=ubuntu-cloud-keyring,python-pycurl) => {"failed": true, "item": "ubuntu-cloud-keyring,python-pycurl"}
msg: No package matching 'ubuntu-cloud-keyring' is available
failed: [swift-keystone-01] => (item=ubuntu-cloud-keyring,python-pycurl) => {"failed": true, "item": "ubuntu-cloud-keyring,python-pycurl"}
msg: No package matching 'ubuntu-cloud-keyring' is available
failed: [swift-storage-01] => (item=ubuntu-cloud-keyring,python-pycurl) => {"failed": true, "item": "ubuntu-cloud-keyring,python-pycurl"}
msg: No package matching 'ubuntu-cloud-keyring' is available
failed: [swift-proxy-01] => (item=ubuntu-cloud-keyring,python-pycurl) => {"failed": true, "item": "ubuntu-cloud-keyring,python-pycurl"}
msg: No package matching 'ubuntu-cloud-keyring' is available
failed: [swift-lbssl-01] => (item=ubuntu-cloud-keyring,python-pycurl) => {"failed": true, "item": "ubuntu-cloud-keyring,python-pycurl"}
msg: No package matching 'ubuntu-cloud-keyring' is available
changed: [swift-package-cache-01] => (item=ubuntu-cloud-keyring,python-pycurl)

Followed by:

GATHERING FACTS ***************************************************************
FATAL: no hosts matched or all hosts have already failed -- aborting

TASK: [lbssl | install required packages for lbssl] ***************************
FATAL: no hosts matched or all hosts have already failed -- aborting
ccollicutt commented 10 years ago

So no changes and thus the vagrant box used will be the precise64 image? I wonder if apt needs to be updated before it installs packages. Let me try a new run with precies64.

ccollicutt commented 10 years ago

Hi, I just pushed a new version with the all.example having the right setup for the interfaces, was using ansible_eth0 instead of ansible_eth1 which won't work in a vagrant environment. Sorry about that, that is a pretty huge bug. Thanks so much for trying this out.

Please give it another shot.

brainstorm commented 10 years ago

Now it goes through but there seem to be some connectivity issues with the ssl load balancer and the proxy host (vagrant destroyed and vagrant upped three times to discard transient failures, got same results :-/):

GATHERING FACTS ***************************************************************
FATAL: no hosts matched or all hosts have already failed -- aborting

TASK: [lbssl | install required packages for lbssl] ***************************
FATAL: no hosts matched or all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/Users/roman/site.retry

swift-keystone-01          : ok=12   changed=10   unreachable=0    failed=0
swift-lbssl-01             : ok=0    changed=0    unreachable=1    failed=0
swift-package-cache-01     : ok=17   changed=13   unreachable=0    failed=0
swift-proxy-01             : ok=0    changed=0    unreachable=1    failed=0
swift-storage-01           : ok=12   changed=10   unreachable=0    failed=0
swift-storage-02           : ok=12   changed=10   unreachable=0    failed=0
swift-storage-03           : ok=12   changed=9    unreachable=0    failed=0
ccollicutt commented 10 years ago

If the node is unreachable by ansible it probably can't ssh into it. I suppose it's possible that the playbook is causing that somehow...can you do "ansible -vvvv -m ping all" and give me the results?

brainstorm commented 10 years ago

Sure, here you go:

$ ansible -vvvv -m ping all
<192.168.100.50> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.100.200> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.100.201> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.100.202> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.100.200> REMOTE_MODULE ping
<192.168.100.50> REMOTE_MODULE ping
<192.168.100.20> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.100.201> REMOTE_MODULE ping
<192.168.100.202> REMOTE_MODULE ping
<192.168.100.20> REMOTE_MODULE ping
<192.168.100.200> EXEC ['ssh', '-C', '-tt', '-vvv', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/roman/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'IdentityFile="/Users/roman/.vagrant.d/insecure_private_key"', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=vagrant', '-o', 'ConnectTimeout=10', '192.168.100.200', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1408435368.21-116143308016256 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1408435368.21-116143308016256 && echo $HOME/.ansible/tmp/ansible-tmp-1408435368.21-116143308016256'"]
<192.168.100.20> EXEC ['ssh', '-C', '-tt', '-vvv', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/roman/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'IdentityFile="/Users/roman/.vagrant.d/insecure_private_key"', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=vagrant', '-o', 'ConnectTimeout=10', '192.168.100.20', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1408435368.21-111987472289220 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1408435368.21-111987472289220 && echo $HOME/.ansible/tmp/ansible-tmp-1408435368.21-111987472289220'"]
<192.168.100.202> EXEC ['ssh', '-C', '-tt', '-vvv', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/roman/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'IdentityFile="/Users/roman/.vagrant.d/insecure_private_key"', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=vagrant', '-o', 'ConnectTimeout=10', '192.168.100.202', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1408435368.21-86965512823400 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1408435368.21-86965512823400 && echo $HOME/.ansible/tmp/ansible-tmp-1408435368.21-86965512823400'"]
<192.168.100.201> EXEC ['ssh', '-C', '-tt', '-vvv', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/roman/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'IdentityFile="/Users/roman/.vagrant.d/insecure_private_key"', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=vagrant', '-o', 'ConnectTimeout=10', '192.168.100.201', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1408435368.21-89585220632314 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1408435368.21-89585220632314 && echo $HOME/.ansible/tmp/ansible-tmp-1408435368.21-89585220632314'"]
<192.168.100.50> EXEC ['ssh', '-C', '-tt', '-vvv', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/roman/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'IdentityFile="/Users/roman/.vagrant.d/insecure_private_key"', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=vagrant', '-o', 'ConnectTimeout=10', '192.168.100.50', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1408435368.21-275888579478621 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1408435368.21-275888579478621 && echo $HOME/.ansible/tmp/ansible-tmp-1408435368.21-275888579478621'"]
swift-package-cache-01 | FAILED => SSH encountered an unknown error. The output was:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/roman/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/Users/roman/.ansible/cp/ansible-ssh-192.168.100.20-22-vagrant" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.100.20 [192.168.100.20] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: connect to address 192.168.100.20 port 22: Operation timed out
ssh: connect to host 192.168.100.20 port 22: Operation timed out

swift-storage-02 | FAILED => SSH encountered an unknown error. The output was:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/roman/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/Users/roman/.ansible/cp/ansible-ssh-192.168.100.201-22-vagrant" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.100.201 [192.168.100.201] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: connect to address 192.168.100.201 port 22: Operation timed out
ssh: connect to host 192.168.100.201 port 22: Operation timed out

swift-storage-01 | FAILED => SSH encountered an unknown error. The output was:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/roman/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/Users/roman/.ansible/cp/ansible-ssh-192.168.100.200-22-vagrant" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.100.200 [192.168.100.200] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: connect to address 192.168.100.200 port 22: Operation timed out
ssh: connect to host 192.168.100.200 port 22: Operation timed out

swift-keystone-01 | FAILED => SSH encountered an unknown error. The output was:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/roman/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/Users/roman/.ansible/cp/ansible-ssh-192.168.100.50-22-vagrant" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.100.50 [192.168.100.50] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: connect to address 192.168.100.50 port 22: Operation timed out
ssh: connect to host 192.168.100.50 port 22: Operation timed out

swift-storage-03 | FAILED => SSH encountered an unknown error. The output was:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/roman/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/Users/roman/.ansible/cp/ansible-ssh-192.168.100.202-22-vagrant" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.100.202 [192.168.100.202] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: connect to address 192.168.100.202 port 22: Operation timed out
ssh: connect to host 192.168.100.202 port 22: Operation timed out

<192.168.100.30> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.100.100> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.100.30> REMOTE_MODULE ping
<192.168.100.100> REMOTE_MODULE ping
<192.168.100.30> EXEC ['ssh', '-C', '-tt', '-vvv', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/roman/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'IdentityFile="/Users/roman/.vagrant.d/insecure_private_key"', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=vagrant', '-o', 'ConnectTimeout=10', '192.168.100.30', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1408435378.24-151851194625521 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1408435378.24-151851194625521 && echo $HOME/.ansible/tmp/ansible-tmp-1408435378.24-151851194625521'"]
<192.168.100.100> EXEC ['ssh', '-C', '-tt', '-vvv', '-o', 'ControlMaster=auto', '-o', 'ControlPersist=60s', '-o', 'ControlPath=/Users/roman/.ansible/cp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'IdentityFile="/Users/roman/.vagrant.d/insecure_private_key"', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey', '-o', 'PasswordAuthentication=no', '-o', 'User=vagrant', '-o', 'ConnectTimeout=10', '192.168.100.100', "/bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1408435378.24-174278524628093 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1408435378.24-174278524628093 && echo $HOME/.ansible/tmp/ansible-tmp-1408435378.24-174278524628093'"]
swift-proxy-01 | FAILED => SSH encountered an unknown error. The output was:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/roman/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/Users/roman/.ansible/cp/ansible-ssh-192.168.100.100-22-vagrant" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.100.100 [192.168.100.100] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: connect to address 192.168.100.100 port 22: Operation timed out
ssh: connect to host 192.168.100.100 port 22: Operation timed out

swift-lbssl-01 | FAILED => SSH encountered an unknown error. The output was:
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/roman/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/Users/roman/.ansible/cp/ansible-ssh-192.168.100.30-22-vagrant" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.100.30 [192.168.100.30] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: connect to address 192.168.100.30 port 22: Operation timed out
ssh: connect to host 192.168.100.30 port 22: Operation timed out
brainstorm commented 10 years ago

Apparently this is a common issue with Vagrant+Ansible+SSH's control socket:

http://stackoverflow.com/questions/22232509/vagrant-provision-works-but-i-cannot-send-an-ad-hoc-command-with-ansible

brainstorm commented 10 years ago

Scratch the unreachable part, apparently some machines were in Shutdown state at some point after being instantiated (even after trying three times in a row, don't ask me why). Today it worked but there are some small warnings to look after:

TASK: [storage | check if account.ring.gz already exists] *********************
failed: [swift-storage-01] => {"changed": true, "cmd": ["ls", "/etc/swift/account.ring.gz"], "delta": "0:00:00.005261", "end": "2014-08-19 07:02:29.901344", "rc": 2, "start": "2014-08-19 07:02:29.896083", "stdout_lines": []}
stderr: ls: cannot access /etc/swift/account.ring.gz: No such file or directory
...ignoring

Thanks for your help! Closing...

brainstorm commented 10 years ago

Oh, also scrap my last comment, I see that this check is to assist with re-runs, smart.