search

seantis / suitable

An Ansible API for humans.
GNU General Public License v3.0
171 stars 23 forks source link

Compatibility between Celery and suitable ？ #31

Closed X-Mars closed 5 years ago

X-Mars commented 5 years ago

dear sir, I got some error when use suitable by Celery。

This is my code

from __future__ import absolute_import, unicode_literals
from celery import shared_task
from action.vdsd_action import VdsdAction

@shared_task
def restart_vdsd(username, k8s_worker, vdsd_name):
    print(k8s_worker)
    print(vdsd_name)
    print(username)
    V = VdsdAction(username=username, k8s_worker=k8s_worker, vdsd_name=vdsd_name)
    log = V.restart_vdsd()
    return log

@shared_task
def restore_vdsd(username, k8s_worker, from_pod_name, vdsd_name_list, datetime_now):
    print(k8s_worker)
    print(vdsd_name_list)
    print(username)
    V = VdsdAction(username=username, k8s_worker=k8s_worker)
    V.restore_vdsd(from_pod_name=from_pod_name, vdsd_name_list=vdsd_name_list, datetime_now=datetime_now)
    return vdsd_name_list
 res = tasks.restore_vdsd.delay(username=username, k8s_worker=worker_name, from_pod_name=from_pod_name, vdsd_name_list=vdsd_dict[worker_name], datetime_now=datetime_now)

    def restore_vdsd(self, from_pod_name, vdsd_name_list, datetime_now):
        print(from_pod_name)
        print(vdsd_name_list)
        copy_data_dir_path = self.make_copy_data_dir_script(datetime_now=datetime_now, hostname=self.hostname, to_pod_str=",".join(vdsd_name_list), from_pod_name=from_pod_name)
        print(copy_data_dir_path)
        host_ip = Server.objects.get(name=self.hostname).ip
        A = AnsibleApi(hostname=host_ip)
        A.restore_vdsd(copy_data_dir_path=copy_data_dir_path)
        return from_pod_name
class AnsibleApi(object):

    def __init__(self, hostname, port=25781, user="sa"):
        self.hostname_full = hostname + ':' + str(port)
        print(self.hostname_full)
        print(user)
        self.user = user
        self.Ansible_api = Api(self.hostname_full, remote_user=user, become=True, become_user='root', verbosity='debug', dry_run=True)
        print(self.Ansible_api)
        self.host_info = {}
        self.stdout = []
        self.ip = hostname

....

    def restore_vdsd(self, copy_data_dir_path):
        print("restore_vdsd")
        print(copy_data_dir_path)
        print(self.Ansible_api.servers)
        print(self.Ansible_api.options)
        print(subprocess.getoutput('whoami'))
        hostname = self.Ansible_api.command(name='date')
        print(hostname)
        print(hostname.msg())
        print(hostname.stdout())

This is logs

[2019-08-29 17:37:08,944: WARNING/ForkPoolWorker-1] /tmp/pycharm_project_804/tmp/qd-worker4/copy_data_dir_script/copy_data_dir_script.py
[2019-08-29 17:37:08,971: WARNING/ForkPoolWorker-1] 2.2.2.2:25781
[2019-08-29 17:37:08,971: WARNING/ForkPoolWorker-1] sa
[2019-08-29 17:37:09,317: WARNING/ForkPoolWorker-1] <suitable.api.Api object at 0x7f7179c43550>
[2019-08-29 17:37:09,576: WARNING/ForkPoolWorker-1] restore_vdsd
[2019-08-29 17:37:09,576: WARNING/ForkPoolWorker-1] /tmp/pycharm_project_804/tmp/qd-worker4/copy_data_dir_script/copy_data_dir_script.py
[2019-08-29 17:37:09,577: WARNING/ForkPoolWorker-1] ['2.2.2.2:25781']
[2019-08-29 17:37:09,577: WARNING/ForkPoolWorker-1] <suitable.utils.options_as_class.<locals>.Options object at 0x7f71790f5f50>
[2019-08-29 17:37:09,586: WARNING/ForkPoolWorker-1] sa
[2019-08-29 17:37:09,866: INFO/ForkPoolWorker-1] running - user: {'name': 'test'}
[2019-08-29 17:37:10,306: WARNING/ForkPoolWorker-1] META: ran handlers
[2019-08-29 17:37:10,316: WARNING/ForkPoolWorker-1] META: ran handlers
[2019-08-29 17:37:10,316: WARNING/ForkPoolWorker-1] META: ran handlers
[2019-08-29 17:37:10,317: DEBUG/ForkPoolWorker-1] took 0:00:00.451190 to complete
[2019-08-29 17:37:10,318: WARNING/ForkPoolWorker-1] {'contacted': {}}
[2019-08-29 17:37:10,322: ERROR/ForkPoolWorker-1] Task action.tasks.restore_vdsd[5cad9ae6-d05d-4fd0-8e7d-b5302e169343] raised unexpected: KeyError('None could not be contacted')
Traceback (most recent call last):
  File "/usr/local/python3.7.4/lib/python3.7/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/python3.7.4/lib/python3.7/site-packages/celery/app/trace.py", line 648, in __protected_call__
    return self.run(*args, **kwargs)
  File "/tmp/pycharm_project_804/action/tasks.py", line 28, in restore_vdsd
    V.restore_vdsd(from_pod_name=from_pod_name, vdsd_name_list=vdsd_name_list, datetime_now=datetime_now)
  File "/tmp/pycharm_project_804/action/vdsd_action.py", line 103, in restore_vdsd
    A.restore_vdsd(copy_data_dir_path=copy_data_dir_path)
  File "/tmp/pycharm_project_804/action/ansible_api.py", line 110, in restore_vdsd
    print(hostname.msg())
  File "/usr/local/python3.7.4/lib/python3.7/site-packages/suitable/runner_results.py", line 17, in <lambda>
    return lambda server=None: self.acquire(server, key)
  File "/usr/local/python3.7.4/lib/python3.7/site-packages/suitable/runner_results.py", line 27, in acquire
    raise KeyError("{} could not be contacted".format(server))
KeyError: 'None could not be contacted'

It is ok when i run suitable without Celery.

Celery: 4.3.0
ansible: 2.8.3

Thanks sir

Your x-mars

href commented 5 years ago

Looks like you're hitting the same error as in #30. Whatever you did there to fix, should work here too. KeyError: 'None could not be contacted' means that Suitable tried to contact a server with the value of None, which of course cannot possibly exist.

X-Mars commented 5 years ago

It is ok when i run suitable without Celery, and the server is reachable.

code 

                # res = tasks.restore_vdsd.delay(username=username, k8s_worker=worker_name, from_pod_name=from_pod_name, vdsd_name_list=vdsd_dict[worker_name], datetime_now=datetime_now)
                res = tasks.restore_vdsd(username=username, k8s_worker=worker_name, from_pod_name=from_pod_name, vdsd_name_list=vdsd_dict[worker_name], datetime_now=datetime_now)
# @shared_task
def restore_vdsd(username, k8s_worker, from_pod_name, vdsd_name_list, datetime_now):
    print(k8s_worker)
    print(vdsd_name_list)
    print(username)
    V = VdsdAction(username=username, k8s_worker=k8s_worker)
    V.restore_vdsd(from_pod_name=from_pod_name, vdsd_name_list=vdsd_name_list, datetime_now=datetime_now)
    return vdsd_name_list
    def restore_vdsd(self, from_pod_name, vdsd_name_list, datetime_now):
        print(from_pod_name)
        print(vdsd_name_list)
        copy_data_dir_path = self.make_copy_data_dir_script(datetime_now=datetime_now, hostname=self.hostname, to_pod_str=",".join(vdsd_name_list), from_pod_name=from_pod_name)
        print(copy_data_dir_path)
        host_ip = Server.objects.get(name=self.hostname).ip
        A = AnsibleApi(hostname=host_ip)
        A.restore_vdsd(copy_data_dir_path=copy_data_dir_path)
        return from_pod_name
<suitable.utils.options_as_class.<locals>.Options object at 0x7fef2b4dea90>
sa
META: ran handlers
<2.2.2.2> ESTABLISH SSH CONNECTION FOR USER: sa
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 2.2.2.2 '/bin/sh -c '"'"'echo ~sa && sleep 0'"'"''
<2.2.2.2> (0, b'/home/sa\n', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/sa/.ansible/cp/fc94eacab8" does not exist\r\ndebug2: resolving "2.2.2.2" port 25781\r\ndebug2: ssh_connect_direct: needpriv 0\r\ndebug1: Connecting to 2.2.2.2 [2.2.2.2] port 25781.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 10000 ms remain after connect\r\ndebug1: identity file /home/sa/.ssh/id_rsa type 1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /home/sa/.ssh/id_rsa-cert type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /home/sa/.ssh/id_dsa type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /home/sa/.ssh/id_dsa-cert type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /home/sa/.ssh/id_ecdsa type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /home/sa/.ssh/id_ecdsa-cert type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /home/sa/.ssh/id_ed25519 type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /home/sa/.ssh/id_ed25519-cert type -1\r\ndebug1: Enabling compatibility mode for protocol 2.0\r\ndebug1: Local version string SSH-2.0-OpenSSH_7.4\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_7.4\r\ndebug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to 2.2.2.2:25781 as \'sa\'\r\ndebug3: put_host_port: [2.2.2.2]:25781\r\ndebug3: hostkeys_foreach: reading file "/home/sa/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/sa/.ssh/known_hosts:2\r\ndebug3: load_hostkeys: loaded 1 keys from [2.2.2.2]:25781\r\ndebug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c\r\ndebug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss\r\ndebug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: zlib@openssh.com,zlib,none\r\ndebug2: compression stoc: zlib@openssh.com,zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1\r\ndebug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,zlib@openssh.com\r\ndebug2: compression stoc: none,zlib@openssh.com\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com\r\ndebug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:fHssRxKlis8jK5/XOOfLx8DLCkdcDZHHhNZGzcncRqo\r\ndebug3: put_host_port: [2.2.2.2]:25781\r\ndebug3: put_host_port: [2.2.2.2]:25781\r\ndebug3: hostkeys_foreach: reading file "/home/sa/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/sa/.ssh/known_hosts:2\r\ndebug3: load_hostkeys: loaded 1 keys from [2.2.2.2]:25781\r\ndebug3: hostkeys_foreach: reading file "/home/sa/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/sa/.ssh/known_hosts:2\r\ndebug3: load_hostkeys: loaded 1 keys from [2.2.2.2]:25781\r\ndebug1: Host \'[2.2.2.2]:25781\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/sa/.ssh/known_hosts:2\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey after 134217728 blocks\r\ndebug2: key: /home/sa/.ssh/id_rsa (0x56368ae25520)\r\ndebug2: key: /home/sa/.ssh/id_dsa ((nil))\r\ndebug2: key: /home/sa/.ssh/id_ecdsa ((nil))\r\ndebug2: key: /home/sa/.ssh/id_ed25519 ((nil))\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: preferred gssapi-with-mic,gssapi-keyex,hostbased,publickey\r\ndebug3: authmethod_lookup gssapi-with-mic\r\ndebug3: remaining preferred: gssapi-keyex,hostbased,publickey\r\ndebug3: authmethod_is_enabled gssapi-with-mic\r\ndebug1: Next authentication method: gssapi-with-mic\r\ndebug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: KEYRING:persistent:1000)\n\r\ndebug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: KEYRING:persistent:1000)\n\r\ndebug2: we did not send a packet, disable method\r\ndebug3: authmethod_lookup gssapi-keyex\r\ndebug3: remaining preferred: hostbased,publickey\r\ndebug3: authmethod_is_enabled gssapi-keyex\r\ndebug1: Next authentication method: gssapi-keyex\r\ndebug1: No valid Key exchange context\r\ndebug2: we did not send a packet, disable method\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: ,publickey\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Offering RSA public key: /home/sa/.ssh/id_rsa\r\ndebug3: send_pubkey_test\r\ndebug3: send packet: type 50\r\ndebug2: we sent a publickey packet, wait for reply\r\ndebug3: receive packet: type 60\r\ndebug1: Server accepts key: pkalg rsa-sha2-512 blen 279\r\ndebug2: input_userauth_pk_ok: fp SHA256:25egVkP3z10EYIbI9ftd7nB1G1bl8/qtAiGDsYCBiPE\r\ndebug3: sign_and_send_pubkey: RSA SHA256:25egVkP3z10EYIbI9ftd7nB1G1bl8/qtAiGDsYCBiPE\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 52\r\ndebug1: Enabling compression at level 6.\r\ndebug1: Authentication succeeded (publickey).\r\nAuthenticated to 2.2.2.2 ([2.2.2.2]:25781).\r\ndebug1: setting up multiplex master socket\r\ndebug3: muxserver_listen: temporary control path /home/sa/.ansible/cp/fc94eacab8.voKtszLWEpZvvLoW\r\ndebug2: fd 4 setting O_NONBLOCK\r\ndebug3: fd 4 is O_NONBLOCK\r\ndebug3: fd 4 is O_NONBLOCK\r\ndebug1: channel 0: new [/home/sa/.ansible/cp/fc94eacab8]\r\ndebug3: muxserver_listen: mux listener channel 0 fd 4\r\ndebug2: fd 3 setting TCP_NODELAY\r\ndebug3: ssh_packet_set_tos: set IP_TOS 0x08\r\ndebug1: control_persist_detach: backgrounding master process\r\ndebug2: control_persist_detach: background process is 9337\r\ndebug2: fd 4 setting O_NONBLOCK\r\ndebug1: forking to background\r\ndebug1: Entering interactive session.\r\ndebug1: pledge: id\r\ndebug2: set_control_persist_exit_time: schedule exit in 60 seconds\r\ndebug1: multiplexing control connection\r\ndebug2: fd 5 setting O_NONBLOCK\r\ndebug3: fd 5 is O_NONBLOCK\r\ndebug1: channel 1: new [mux-control]\r\ndebug3: channel_post_mux_listener: new mux channel 1 fd 5\r\ndebug3: mux_master_read_cb: channel 1: hello sent\r\ndebug2: set_control_persist_exit_time: cancel scheduled exit\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x00000001 len 4\r\ndebug2: process_mux_master_hello: channel 1 slave version 4\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x10000004 len 4\r\ndebug2: process_mux_alive_check: channel 1: alive check\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x10000002 len 93\r\ndebug2: process_mux_new_session: channel 1: request tty 0, X 0, agent 0, subsys 0, term "vt100", cmd "/bin/sh -c \'echo ~sa && sleep 0\'", env 1\r\ndebug3: process_mux_new_session: got fds stdin 6, stdout 7, stderr 8\r\ndebug2: fd 7 setting O_NONBLOCK\r\ndebug2: fd 8 setting O_NONBLOCK\r\ndebug1: channel 2: new [client-session]\r\ndebug2: process_mux_new_session: channel_new: 2 linked to control channel 1\r\ndebug2: channel 2: send open\r\ndebug3: send packet: type 90\r\ndebug3: receive packet: type 80\r\ndebug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0\r\ndebug3: receive packet: type 91\r\ndebug2: callback start\r\ndebug2: client_session2_setup: id 2\r\ndebug1: Sending environment.\r\ndebug1: Sending env LANG = en_US.UTF-8\r\ndebug2: channel 2: request env confirm 0\r\ndebug3: send packet: type 98\r\ndebug1: Sending command: /bin/sh -c \'echo ~sa && sleep 0\'\r\ndebug2: channel 2: request exec confirm 1\r\ndebug3: send packet: type 98\r\ndebug3: mux_session_confirm: sending success reply\r\ndebug2: callback done\r\ndebug2: channel 2: open confirm rwindow 0 rmax 32768\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: channel 2: rcvd adjust 2097152\r\ndebug3: receive packet: type 99\r\ndebug2: channel_input_status_confirm: type 99 id 2\r\ndebug2: exec request accepted on channel 2\r\ndebug3: receive packet: type 96\r\ndebug2: channel 2: rcvd eof\r\ndebug2: channel 2: output open -> drain\r\ndebug2: channel 2: obuf empty\r\ndebug2: channel 2: close_write\r\ndebug2: channel 2: output drain -> closed\r\ndebug3: receive packet: type 98\r\ndebug1: client_input_channel_req: channel 2 rtype exit-status reply 0\r\ndebug3: mux_exit_message: channel 2: exit message, exitval 0\r\ndebug3: receive packet: type 98\r\ndebug1: client_input_channel_req: channel 2 rtype eow@openssh.com reply 0\r\ndebug2: channel 2: rcvd eow\r\ndebug2: channel 2: close_read\r\ndebug2: channel 2: input open -> closed\r\ndebug3: receive packet: type 97\r\ndebug2: channel 2: rcvd close\r\ndebug3: channel 2: will not send data after close\r\ndebug2: channel 2: send close\r\ndebug3: send packet: type 97\r\ndebug2: channel 2: is dead\r\ndebug2: channel 2: gc: notify user\r\ndebug3: mux_master_session_cleanup_cb: entering for channel 2\r\ndebug2: channel 1: rcvd close\r\ndebug2: channel 1: output open -> drain\r\ndebug2: channel 1: close_read\r\ndebug2: channel 1: input open -> closed\r\ndebug2: channel 2: gc: user detached\r\ndebug2: channel 2: is dead\r\ndebug2: channel 2: garbage collecting\r\ndebug1: channel 2: free: client-session, nchannels 3\r\ndebug3: channel 2: status: The following connections are open:\r\n  #1 mux-control (t16 r-1 i3/0 o1/16 fd 5/5 cc -1)\r\n  #2 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cc -1)\r\n\r\ndebug2: channel 1: obuf empty\r\ndebug2: channel 1: close_write\r\ndebug2: channel 1: output drain -> closed\r\ndebug2: channel 1: is dead (local)\r\ndebug2: channel 1: gc: notify user\r\ndebug3: mux_master_control_cleanup_cb: entering for channel 1\r\ndebug2: channel 1: gc: user detached\r\ndebug2: channel 1: is dead (local)\r\ndebug2: channel 1: garbage collecting\r\ndebug1: channel 1: free: mux-control, nchannels 2\r\ndebug3: channel 1: status: The following connections are open:\r\n  #1 mux-control (t16 r-1 i3/0 o3/0 fd 5/5 cc -1)\r\n\r\ndebug2: set_control_persist_exit_time: schedule exit in 60 seconds\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<2.2.2.2> ESTABLISH SSH CONNECTION FOR USER: sa
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 2.2.2.2 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764 `" && echo ansible-tmp-1567071579.129729-280378901863764="` echo /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764 `" ) && sleep 0'"'"''
<2.2.2.2> (0, b'ansible-tmp-1567071579.129729-280378901863764=/home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764\n', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/basic.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/_text.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/distro/__init__.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/parsing/__init__.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/parameters.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/text/__init__.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/text/formatters.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/_utils.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/six/__init__.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/validation.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/sys_info.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/process.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/__init__.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/pycompat24.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/file.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/_json_compat.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/common/collections.py
Using module_utils file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/module_utils/distro/_distro.py
<2.2.2.2> Attempting python interpreter discovery
<2.2.2.2> ESTABLISH SSH CONNECTION FOR USER: sa
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 2.2.2.2 '/bin/sh -c '"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'/usr/bin/python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.5'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/libexec/platform-python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"''
<2.2.2.2> (0, b'PLATFORM\nLinux\nFOUND\n/usr/bin/python\n/usr/bin/python2.7\n/usr/libexec/platform-python\n/usr/bin/python\nENDFOUND\n', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<2.2.2.2> ESTABLISH SSH CONNECTION FOR USER: sa
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 2.2.2.2 '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
<2.2.2.2> (0, b'{"osrelease_content": "NAME=\\"CentOS Linux\\"\\nVERSION=\\"7 (Core)\\"\\nID=\\"centos\\"\\nID_LIKE=\\"rhel fedora\\"\\nVERSION_ID=\\"7\\"\\nPRETTY_NAME=\\"CentOS Linux 7 (Core)\\"\\nANSI_COLOR=\\"0;31\\"\\nCPE_NAME=\\"cpe:/o:centos:centos:7\\"\\nHOME_URL=\\"https://www.centos.org/\\"\\nBUG_REPORT_URL=\\"https://bugs.centos.org/\\"\\n\\nCENTOS_MANTISBT_PROJECT=\\"CentOS-7\\"\\nCENTOS_MANTISBT_PROJECT_VERSION=\\"7\\"\\nREDHAT_SUPPORT_PRODUCT=\\"centos\\"\\nREDHAT_SUPPORT_PRODUCT_VERSION=\\"7\\"\\n\\n", "platform_dist_result": ["centos", "7.6.1810", "Core"]}\n', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
Using module file /usr/local/python3.7.4/lib/python3.7/site-packages/ansible/modules/system/user.py
<2.2.2.2> PUT /home/sa/.ansible/tmp/ansible-local-92218iw9ig47/tmp9rau5u6w TO /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764/AnsiballZ_user.py
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 '[2.2.2.2]'
<2.2.2.2> (0, b'sftp> put /home/sa/.ansible/tmp/ansible-local-92218iw9ig47/tmp9rau5u6w /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764/AnsiballZ_user.py\n', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "posix-rename@openssh.com" revision 1\r\ndebug2: Server supports extension "statvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "fstatvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "hardlink@openssh.com" revision 1\r\ndebug2: Server supports extension "fsync@openssh.com" revision 1\r\ndebug3: Sent message fd 3 T:16 I:1\r\ndebug3: SSH_FXP_REALPATH . -> /home/sa size 0\r\ndebug3: Looking up /home/sa/.ansible/tmp/ansible-local-92218iw9ig47/tmp9rau5u6w\r\ndebug3: Sent message fd 3 T:17 I:2\r\ndebug3: Received stat reply T:101 I:2\r\ndebug1: Couldn\'t stat remote file: No such file or directory\r\ndebug3: Sent message SSH2_FXP_OPEN I:3 P:/home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764/AnsiballZ_user.py\r\ndebug3: Sent message SSH2_FXP_WRITE I:4 O:0 S:32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 4 32768 bytes at 0\r\ndebug3: Sent message SSH2_FXP_WRITE I:5 O:32768 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:65536 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:7 O:98304 S:26768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 5 32768 bytes at 32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 32768 bytes at 65536\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 7 26768 bytes at 98304\r\ndebug3: Sent message SSH2_FXP_CLOSE I:4\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<2.2.2.2> ESTABLISH SSH CONNECTION FOR USER: sa
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 2.2.2.2 '/bin/sh -c '"'"'chmod u+x /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764/ /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764/AnsiballZ_user.py && sleep 0'"'"''
<2.2.2.2> (0, b'', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<2.2.2.2> ESTABLISH SSH CONNECTION FOR USER: sa
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 -tt 2.2.2.2 '/bin/sh -c '"'"'sudo -H -S -n  -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-wpdwncidjdrivsrsnetzopxcojnvjhth ; /usr/bin/python /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764/AnsiballZ_user.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<2.2.2.2> (0, b'\r\n{"comment": "", "shell": "/bin/bash", "group": 1001, "uid": 1001, "changed": true, "create_home": true, "system": false, "state": "present", "invocation": {"module_args": {"comment": null, "ssh_key_bits": 0, "update_password": "always", "non_unique": false, "force": false, "ssh_key_type": "rsa", "create_home": true, "password_lock": null, "ssh_key_passphrase": null, "uid": null, "home": null, "append": false, "skeleton": null, "ssh_key_comment": "ansible-generated on qd-worker4", "group": null, "system": false, "state": "present", "role": null, "hidden": null, "local": null, "authorization": null, "profile": null, "shell": null, "expires": null, "ssh_key_file": null, "groups": null, "move_home": false, "password": null, "name": "test", "seuser": null, "remove": false, "login_class": null, "generate_ssh_key": null}}, "home": "/home/test", "name": "test"}\r\n', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to 2.2.2.2 closed.\r\n')
<2.2.2.2> ESTABLISH SSH CONNECTION FOR USER: sa
<2.2.2.2> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<2.2.2.2> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=25781)
<2.2.2.2> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<2.2.2.2> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="sa")
<2.2.2.2> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<2.2.2.2> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/sa/.ansible/cp/fc94eacab8)
<2.2.2.2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o Port=25781 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="sa"' -o ConnectTimeout=10 -o ControlPath=/home/sa/.ansible/cp/fc94eacab8 2.2.2.2 '/bin/sh -c '"'"'rm -f -r /home/sa/.ansible/tmp/ansible-tmp-1567071579.129729-280378901863764/ > /dev/null 2>&1 && sleep 0'"'"''
<2.2.2.2> (0, b'', b'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 9339\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
META: ran handlers
META: ran handlers
{'contacted': {'2.2.2.2:25781': {'comment': '', 'shell': '/bin/bash', 'group': 1001, 'uid': 1001, 'changed': True, 'create_home': True, 'system': False, 'state': 'present', 'invocation': {'module_args': {'comment': None, 'ssh_key_bits': 0, 'update_password': 'always', 'non_unique': False, 'force': False, 'ssh_key_type': 'rsa', 'create_home': True, 'password_lock': None, 'ssh_key_passphrase': None, 'uid': None, 'home': None, 'append': False, 'skeleton': None, 'ssh_key_comment': 'ansible-generated on qd-worker4', 'group': None, 'system': False, 'state': 'present', 'role': None, 'hidden': None, 'local': None, 'authorization': None, 'profile': None, 'shell': None, 'expires': None, 'ssh_key_file': None, 'groups': None, 'move_home': False, 'password': None, 'name': 'test', 'seuser': None, 'remove': False, 'login_class': None, 'generate_ssh_key': None}}, 'home': '/home/test', 'name': 'test', 'ansible_facts': {'discovered_interpreter_python': '/usr/bin/python'}, '_ansible_no_log': False}}}
href commented 5 years ago

My guess would be that Server.objects.get(name=self.hostname).ip returns None. In either case, this is probably not a Suitable issue.

X-Mars commented 5 years ago

Server.objects.get(name=self.hostname).ip can got ip 😢 I will keep try，and return message here if i got something

href commented 5 years ago

You can always add a breakpoint on the following line and see where the None value comes from, by inspecting the call stack: https://github.com/seantis/suitable/blob/master/suitable/api.py#L163

X-Mars commented 5 years ago

I have rewrite https://github.com/seantis/suitable/blob/master/suitable/api.py#L163 ，and I can got self.servers

code: 


    def __init__(
            self, servers,
            ignore_unreachable=False,
            ignore_errors=False,
            host_key_checking=True,
            sudo=False,
            dry_run=False,
            verbosity='info',
            environment=None,
            strategy=None,
            **options
    ):
        print("====================================================================================")
        print(servers)
        if isinstance(servers, string_types):
            self.servers = servers.split(u' ')
        else:
            self.servers = list(servers)

        print(self.servers)
print("====================================================================================")

logs : 

[2019-08-30 16:52:42,828: DEBUG/MainProcess] Task accepted: action.tasks.restore_vdsd[d41bc18e-19cb-443e-9d38-72396d9e099d] pid:12625
[2019-08-30 16:52:42,852: WARNING/ForkPoolWorker-1] /tmp/pycharm_project_804/tmp/cz-worker13/copy_data_dir_script/copy_data_dir_script.py
[2019-08-30 16:52:42,888: WARNING/ForkPoolWorker-1] 2.2.2.2
[2019-08-30 16:52:42,888: WARNING/ForkPoolWorker-1] 2.2.2.2:25781
[2019-08-30 16:52:42,888: WARNING/ForkPoolWorker-1] sa
[2019-08-30 16:52:42,889: WARNING/ForkPoolWorker-1] ====================================================================================
[2019-08-30 16:52:42,889: WARNING/ForkPoolWorker-1] 2.2.2.2:25781
[2019-08-30 16:52:42,889: WARNING/ForkPoolWorker-1] ['2.2.2.2:25781']
[2019-08-30 16:52:43,351: WARNING/ForkPoolWorker-1] ====================================================================================
[2019-08-30 16:52:43,352: WARNING/ForkPoolWorker-1] <action.ansible_api_test.NewApi object at 0x7fcc849e9950>
[2019-08-30 16:52:43,352: WARNING/ForkPoolWorker-1] restore_vdsd
[2019-08-30 16:52:43,352: WARNING/ForkPoolWorker-1] /tmp/pycharm_project_804/tmp/cz-worker13/copy_data_dir_script/copy_data_dir_script.py
[2019-08-30 16:52:43,352: WARNING/ForkPoolWorker-1] ['2.2.2.2:25781']
[2019-08-30 16:52:43,353: WARNING/ForkPoolWorker-1] <suitable.utils.options_as_class.<locals>.Options object at 0x7fcc849e96d0>
[2019-08-30 16:52:43,368: WARNING/ForkPoolWorker-1] sa
[2019-08-30 16:52:43,652: INFO/ForkPoolWorker-1] running - command: hostname
[2019-08-30 16:52:44,096: WARNING/ForkPoolWorker-1] META: ran handlers
[2019-08-30 16:52:44,107: WARNING/ForkPoolWorker-1] META: ran handlers
[2019-08-30 16:52:44,108: WARNING/ForkPoolWorker-1] META: ran handlers
[2019-08-30 16:52:44,109: DEBUG/ForkPoolWorker-1] took 0:00:00.456762 to complete
[2019-08-30 16:52:44,110: WARNING/ForkPoolWorker-1] {'contacted': {}}
[2019-08-30 16:52:44,117: ERROR/ForkPoolWorker-1] Task action.tasks.restore_vdsd[d41bc18e-19cb-443e-9d38-72396d9e099d] raised unexpec
href commented 5 years ago

From what I can tell Suitable is not doing anything wrong here. This looks to me like you have an issue in your code that you need to figure out. I'm afraid I don't have too much time on my hands so I can't look into this for you.

X-Mars commented 5 years ago

I have change it to ansible cli. ☹️ Thank you for your selfless dedication I close it now.