"CHECKING SSH CONNECTIONS" banner, log output and function name are misleading

[inhumantsar]

The banner and log output of deploy.check_ping() suggest that kargo-cli is attempting to SSH into the hosts, however it seems to be doing an ICMP ping. Not all security configurations allow ICMP to hosts, including the AWS default security groups.

This function should either log its actions differently or attempt to a socket connection to the SSH port rather than ping the host.

[ant31]

Good point, but it is not icmp ping. It uses the 'ping' module of ansible, which is not a real ping.

It tries to ssh to the host: http://docs.ansible.com/ansible/ping_module.html

A trivial test module, this module always returns pong on successful contact. It does not make sense in playbooks, but it is useful from /usr/bin/ansible to verify the ability to login and that a usable python is configured. This is NOT ICMP ping, this is just a trivial test module.

[inhumantsar]

Then something else is broken. This is what I attempted:

$ cat ~/kargo/inventory/inventory.cfg
[kube-master]
k8s-atomic-9e79y        
k8s-atomic-g89kj        

[all]
k8s-atomic-9e79y        ansible_ssh_host=10.22.28.85
k8s-atomic-g89kj        ansible_ssh_host=10.22.28.84
k8s-atomic-vmglv        ansible_ssh_host=10.22.28.86

[k8s-cluster:children]
kube-node       
kube-master     

[kube-node]
k8s-atomic-9e79y        
k8s-atomic-g89kj        
k8s-atomic-vmglv        

[etcd]
k8s-atomic-9e79y        
k8s-atomic-g89kj        
k8s-atomic-vmglv        

$ kargo deploy --config dev_config.yml -k ~/.ssh/admin_dev -N 10.175.0.0/16 -u centos --aws --ansible-opts '-e foo=bar -vvv'

CHECKING SSH CONNECTIONS *******************************************************
k8s-atomic-vmglv | UNREACHABLE! => {
    "changed": false, 
    "msg": "Failed to connect to the host via ssh.", 
    "unreachable": true
}
k8s-atomic-9e79y | UNREACHABLE! => {
    "changed": false, 
    "msg": "Failed to connect to the host via ssh.", 
    "unreachable": true
}
k8s-atomic-g89kj | UNREACHABLE! => {
    "changed": false, 
    "msg": "Failed to connect to the host via ssh.", 
    "unreachable": true
}

$ ssh -i ~/.ssh/admin_dev centos@10.22.28.85
Last login: Thu Jun 16 18:35:46 2016 from ip-10-22-0-150.ec2.internal
[centos@ip-10-22-28-85 ~]$ exit
logout
Connection to 10.22.28.85 closed.

[ant31]

can you try: ansible -i ~/kargo/inventory/inventory.cfg -m ping -vvvv all

[inhumantsar]

$ ansible -i ~/kargo/inventory/inventory.cfg -m ping -vvvv all
Using /home/smartin/kargo/ansible.cfg as config file
Loaded callback minimal of type stdout, v2.0
<10.22.28.85> ESTABLISH SSH CONNECTION FOR USER: None
<10.22.28.85> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/smartin/.ansible/cp/ansible-ssh-%h-%p-%r 10.22.28.85 '/bin/sh -c '"'"'LANG=en_CA.UTF-8 LC_ALL=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8 /usr/bin/python && sleep 0'"'"''
<10.22.28.84> ESTABLISH SSH CONNECTION FOR USER: None
<10.22.28.86> ESTABLISH SSH CONNECTION FOR USER: None
<10.22.28.84> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/smartin/.ansible/cp/ansible-ssh-%h-%p-%r 10.22.28.84 '/bin/sh -c '"'"'LANG=en_CA.UTF-8 LC_ALL=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8 /usr/bin/python && sleep 0'"'"''
<10.22.28.86> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/smartin/.ansible/cp/ansible-ssh-%h-%p-%r 10.22.28.86 '/bin/sh -c '"'"'LANG=en_CA.UTF-8 LC_ALL=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8 /usr/bin/python && sleep 0'"'"''
k8s-atomic-vmglv | UNREACHABLE! => {
    "changed": false, 
    "msg": "SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh", 
    "unreachable": true
}
k8s-atomic-9e79y | UNREACHABLE! => {
    "changed": false, 
    "msg": "SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh", 
    "unreachable": true
}
k8s-atomic-g89kj | UNREACHABLE! => {
    "changed": false, 
    "msg": "SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh", 
    "unreachable": true
}

$ ansible -i ~/kargo/inventory/inventory.cfg -m ping -vvvv all -u centos --private-key ~/.ssh/admin_dev
Using /home/smartin/kargo/ansible.cfg as config file
Loaded callback minimal of type stdout, v2.0
<10.22.28.85> ESTABLISH SSH CONNECTION FOR USER: centos
<10.22.28.86> ESTABLISH SSH CONNECTION FOR USER: centos
<10.22.28.84> ESTABLISH SSH CONNECTION FOR USER: centos
<10.22.28.86> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/smartin/.ssh/admin_dev"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=centos -o ConnectTimeout=10 -o ControlPath=/home/smartin/.ansible/cp/ansible-ssh-%h-%p-%r 10.22.28.86 '/bin/sh -c '"'"'LANG=en_CA.UTF-8 LC_ALL=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8 /usr/bin/python && sleep 0'"'"''
<10.22.28.85> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/smartin/.ssh/admin_dev"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=centos -o ConnectTimeout=10 -o ControlPath=/home/smartin/.ansible/cp/ansible-ssh-%h-%p-%r 10.22.28.85 '/bin/sh -c '"'"'LANG=en_CA.UTF-8 LC_ALL=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8 /usr/bin/python && sleep 0'"'"''
<10.22.28.84> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/smartin/.ssh/admin_dev"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=centos -o ConnectTimeout=10 -o ControlPath=/home/smartin/.ansible/cp/ansible-ssh-%h-%p-%r 10.22.28.84 '/bin/sh -c '"'"'LANG=en_CA.UTF-8 LC_ALL=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8 /usr/bin/python && sleep 0'"'"''
k8s-atomic-9e79y | SUCCESS => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "data": null
        }, 
        "module_name": "ping"
    }, 
    "ping": "pong"
}
k8s-atomic-g89kj | SUCCESS => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "data": null
        }, 
        "module_name": "ping"
    }, 
    "ping": "pong"
}
k8s-atomic-vmglv | SUCCESS => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "data": null
        }, 
        "module_name": "ping"
    }, 
    "ping": "pong"
}

[inhumantsar]

check_ping() doesn't use the private key supplied

        cmd = [
            ansible_exec, '--ssh-extra-args', '-o StrictHostKeyChecking=no',
            '-u', '%s' % self.options['ansible_user'],
            '-b', '--become-user=root', '-m', 'ping', 'all',
            '-i', self.inventorycfg
        ]

[ant31]

cc @Smana

[Smana]

@inhumantsar What is your operating system please ? When i use deploy the ssh key is added :

kargo deploy --gce -n calico
Enter passphrase for /home/smana/.ssh/id_rsa: 
Identity added: /home/smana/.ssh/id_rsa (/home/smana/.ssh/id_rsa)

[Smana]

ah that may be when you use th -k option i need to check that.

[Smana]

Could you please try again ?