search

ansible-collections / amazon.aws

Ansible Collection for Amazon AWS
GNU General Public License v3.0
304 stars 334 forks source link

Secrets lookup failing in virtual env: "ERROR! A worker was found in a dead state" #1171

Closed kennedyjosh closed 1 year ago

kennedyjosh commented 1 year ago

Summary

I have a system that will run ansible playbooks from a command-line script. This command line script runs in Python using argparse and will call an ansible playbook using the subprocess package. The problem is, this python script runs in a virtual environment, and that is somehow causing amazon.aws's lookup plugin to crash when I try to fetch a secret from secrets manager.

Below I include instructions to reproduce the issue in a minimal way. If you run the python script without the virtual environment, it works fine.

Issue Type

Bug Report

Component Name

lookup/aws_secret.py

Ansible Version

$ ansible --version
ansible [core 2.13.3]
  config file = /Users/josh/Code/server/ansible.cfg
  configured module search path = ['/Users/josh/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/josh/Code/server/hapy/venv/lib/python3.8/site-packages/ansible
  ansible collection location = /Users/josh/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/josh/Code/server/hapy/venv/bin/ansible
  python version = 3.8.9 (default, Jul 19 2021, 09:37:30) [Clang 13.0.0 (clang-1300.0.27.3)]
  jinja version = 3.1.2
  libyaml = False

Collection Versions

$ ansible-galaxy collection list

# /Users/josh/.ansible/collections/ansible_collections
Collection    Version
------------- -------
amazon.aws    3.4.0  
ansible.posix 1.4.0  
community.aws 3.5.0  

# /Users/josh/Code/server/hapy/venv/lib/python3.8/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    3.4.0  
ansible.netcommon             3.1.0  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.11.0 
arista.eos                    5.0.1  
awx.awx                       21.4.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.3.0  
cisco.aci                     2.2.0  
cisco.asa                     3.1.0  
cisco.dnac                    6.5.3  
cisco.intersight              1.0.19 
cisco.ios                     3.3.0  
cisco.iosxr                   3.3.0  
cisco.ise                     2.5.0  
cisco.meraki                  2.10.1 
cisco.mso                     2.0.0  
cisco.nso                     1.0.3  
cisco.nxos                    3.1.0  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 3.5.0  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.5.0  
community.digitalocean        1.21.0 
community.dns                 2.3.1  
community.docker              2.7.1  
community.fortios             1.0.0  
community.general             5.5.0  
community.google              1.0.0  
community.grafana             1.5.2  
community.hashi_vault         3.2.0  
community.hrobot              1.5.2  
community.libvirt             1.2.0  
community.mongodb             1.4.2  
community.mysql               3.4.0  
community.network             4.0.1  
community.okd                 2.2.0  
community.postgresql          2.2.0  
community.proxysql            1.4.0  
community.rabbitmq            1.2.2  
community.routeros            2.2.1  
community.sap                 1.0.0  
community.sap_libs            1.2.0  
community.skydive             1.0.0  
community.sops                1.3.0  
community.vmware              2.8.0  
community.windows             1.11.0 
community.zabbix              1.8.0  
containers.podman             1.9.4  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.1  
dellemc.openmanage            5.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.19.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.7  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.8.1  
hpe.nimble                    1.1.4  
ibm.qradar                    2.0.0  
ibm.spectrum_virtualize       1.9.0  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.3.0  
inspur.sm                     2.0.0  
junipernetworks.junos         3.1.0  
kubernetes.core               2.3.2  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.19.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.22.0
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.1  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.8.0  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.2.3  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.9.0  
purestorage.fusion            1.0.2  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     2.0.0  
t_systems_mms.icinga_director 1.31.0 
theforeman.foreman            3.4.0  
vmware.vmware_rest            2.2.0  
vyos.vyos                     3.0.1  
wti.remote                    1.0.4

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.18.0
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/josh/Code/server/hapy/venv/lib/python3.8/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.21.0
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/josh/Code/server/hapy/venv/lib/python3.8/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
DEFAULT_HOST_LIST(/Users/josh/Code/server/ansible.cfg) = ['/Users/josh/Code/server/ansible/inventory_aws_ec2.yml']
DISPLAY_SKIPPED_HOSTS(/Users/josh/Code/server/ansible.cfg) = False
HOST_KEY_CHECKING(/Users/josh/Code/server/ansible.cfg) = False
INTERPRETER_PYTHON(/Users/josh/Code/server/ansible.cfg) = python

OS / Environment

macOS 12.5

From virtual environment:

$ bash version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin21)
Copyright (C) 2007 Free Software Foundation, Inc.

$ python3 -V
Python 3.8.9

$ pip freeze
aiodns==2.0.0
aiohttp==3.6.2
ansible==6.3.0
ansible-core==2.13.3
argcomplete==2.0.0
async-timeout==3.0.1
attrs==19.3.0
bcrypt==4.0.0
boto3==1.18.0
botocore==1.21.0
certifi==2020.4.5.1
cffi==1.14.4
chardet==3.0.4
coverage==6.4.4
cryptography==37.0.4
distlib==0.3.6
docutils==0.15.2
fabric==2.5.0
filelock==3.8.0
flake8==5.0.4
idna==2.9
idna-ssl==1.1.0
invoke==1.7.3
Jinja2==3.1.2
jmespath==0.9.5
MarkupSafe==2.1.1
mccabe==0.7.0
multidict==4.7.6
packaging==21.3
paramiko==2.11.0
pathlib2==2.3.7.post1
pexpect==4.8.0
platformdirs==2.5.2
protobuf==4.21.6
ptyprocess==0.7.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycares==3.1.1
pycodestyle==2.9.1
pycparser==2.20
pyflakes==2.5.0
PyNaCl==1.5.0
pyparsing==3.0.9
python-dateutil==2.8.1
PyYAML==6.0
requests==2.23.0
resolvelib==0.8.1
s3transfer==0.5.0
six==1.14.0
typing==3.7.4.1
typing-extensions==3.7.4.2
urllib3==1.25.9
virtualenv==20.16.4
yarl==1.4.2

Steps to Reproduce

h

#!/bin/bash

source venv/bin/activate

exec python3 test.py

h.py

import subprocess

subprocess.run("ansible-playbook h.yml", shell=True)

h.yml

- hosts: new_machines
  tasks:
    - name: Fetch secret
      debug:
        msg: "{{ lookup('amazon.aws.aws_secret', 'my-secret') }}"

On the command line:

$ ./h

Expected Results

I expect my-secret to be printed out as a debug msg from ansible.

Actual Results

Before running this, I modified h.py to call ansible-playbook with -vvvvvv

$ ./h
ansible-playbook [core 2.13.3]
  config file = /Users/josh/Code/server/ansible.cfg
  configured module search path = ['/Users/josh/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/josh/Code/server/hapy/venv/lib/python3.8/site-packages/ansible
  ansible collection location = /Users/josh/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/josh/Code/server/hapy/venv/bin/ansible-playbook
  python version = 3.8.9 (default, Jul 19 2021, 09:37:30) [Clang 13.0.0 (clang-1300.0.27.3)]
  jinja version = 3.1.2
  libyaml = False
Using /Users/josh/Code/server/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /Users/josh/Code/server/ansible/inventory_aws_ec2.yml as it did not pass its verify_file() method
script declined parsing /Users/josh/Code/server/ansible/inventory_aws_ec2.yml as it did not pass its verify_file() method
Loading collection amazon.aws from /Users/josh/.ansible/collections/ansible_collections/amazon/aws
Using inventory plugin 'ansible_collections.amazon.aws.plugins.inventory.aws_ec2' to process inventory source '/Users/josh/Code/server/ansible/inventory_aws_ec2.yml'
Parsed /Users/josh/Code/server/ansible/inventory_aws_ec2.yml inventory source with auto plugin
Loading callback plugin default of type stdout, v2.0 from /Users/josh/Code/server/hapy/venv/lib/python3.8/site-packages/ansible/plugins/callback/default.py
Attempting to use 'default' callback.
Skipping callback 'default', as we already have a stdout callback.
Attempting to use 'junit' callback.
Attempting to use 'minimal' callback.
Skipping callback 'minimal', as we already have a stdout callback.
Attempting to use 'oneline' callback.
Skipping callback 'oneline', as we already have a stdout callback.
Attempting to use 'tree' callback.

PLAYBOOK: h.yml ********************************************************************************************************************************************************************************
Positional arguments: /Users/josh/Code/server/ansible/h.yml
verbosity: 6
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/Users/josh/Code/server/ansible/inventory_aws_ec2.yml',)
forks: 5
1 plays in /Users/josh/Code/server/ansible/h.yml

PLAY [new_machines] *******************************************************************************************************************************************************************************
<52.40.31.173> ESTABLISH SSH CONNECTION FOR USER: None
<52.40.31.173> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ControlPath=/tmp/ansible-ssh-%h-%p-%r)(-o)(ForwardAgent=yes)
<52.40.31.173> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<52.40.31.173> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<52.40.31.173> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<52.40.31.173> SSH: Set ssh_common_args: ()
<52.40.31.173> SSH: Set ssh_extra_args: ()
<52.40.31.173> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 52.40.31.173 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
<52.40.31.173> (0, b'/home/josh\n', b"OpenSSH_8.6p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/josh/.ssh/config\r\ndebug1: /Users/josh/.ssh/config line 1: Applying options for *\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 52.40.31.173 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/josh/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/josh/.ssh/known_hosts2'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 5546\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<52.40.31.173> ESTABLISH SSH CONNECTION FOR USER: None
<52.40.31.173> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ControlPath=/tmp/ansible-ssh-%h-%p-%r)(-o)(ForwardAgent=yes)
<52.40.31.173> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<52.40.31.173> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<52.40.31.173> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<52.40.31.173> SSH: Set ssh_common_args: ()
<52.40.31.173> SSH: Set ssh_extra_args: ()
<52.40.31.173> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 52.40.31.173 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/josh/.ansible/tmp `"&& mkdir "` echo /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097 `" && echo ansible-tmp-1665701298.5003798-5566-235954551572097="` echo /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097 `" ) && sleep 0'"'"''
<52.40.31.173> (0, b'ansible-tmp-1665701298.5003798-5566-235954551572097=/home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097\n', b"OpenSSH_8.6p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/josh/.ssh/config\r\ndebug1: /Users/josh/.ssh/config line 1: Applying options for *\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 52.40.31.173 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/josh/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/josh/.ssh/known_hosts2'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 5546\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
Including module_utils file ansible/__init__.py
Including module_utils file ansible/module_utils/__init__.py
Including module_utils file ansible/module_utils/_text.py
Including module_utils file ansible/module_utils/basic.py
Including module_utils file ansible/module_utils/common/_collections_compat.py
Including module_utils file ansible/module_utils/common/__init__.py
Including module_utils file ansible/module_utils/common/_json_compat.py
Including module_utils file ansible/module_utils/common/_utils.py
Including module_utils file ansible/module_utils/common/arg_spec.py
Including module_utils file ansible/module_utils/common/file.py
Including module_utils file ansible/module_utils/common/locale.py
Including module_utils file ansible/module_utils/common/parameters.py
Including module_utils file ansible/module_utils/common/collections.py
Including module_utils file ansible/module_utils/common/process.py
Including module_utils file ansible/module_utils/common/sys_info.py
Including module_utils file ansible/module_utils/common/text/converters.py
Including module_utils file ansible/module_utils/common/text/__init__.py
Including module_utils file ansible/module_utils/common/text/formatters.py
Including module_utils file ansible/module_utils/common/validation.py
Including module_utils file ansible/module_utils/common/warnings.py
Including module_utils file ansible/module_utils/compat/selectors.py
Including module_utils file ansible/module_utils/compat/__init__.py
Including module_utils file ansible/module_utils/compat/_selectors2.py
Including module_utils file ansible/module_utils/compat/selinux.py
Including module_utils file ansible/module_utils/distro/__init__.py
Including module_utils file ansible/module_utils/distro/_distro.py
Including module_utils file ansible/module_utils/errors.py
Including module_utils file ansible/module_utils/facts/ansible_collector.py
Including module_utils file ansible/module_utils/compat/typing.py
Including module_utils file ansible/module_utils/facts/__init__.py
Including module_utils file ansible/module_utils/facts/collector.py
Including module_utils file ansible/module_utils/facts/compat.py
Including module_utils file ansible/module_utils/facts/default_collectors.py
Including module_utils file ansible/module_utils/facts/hardware/aix.py
Including module_utils file ansible/module_utils/facts/hardware/__init__.py
Including module_utils file ansible/module_utils/facts/hardware/base.py
Including module_utils file ansible/module_utils/facts/hardware/darwin.py
Including module_utils file ansible/module_utils/facts/hardware/dragonfly.py
Including module_utils file ansible/module_utils/facts/hardware/freebsd.py
Including module_utils file ansible/module_utils/facts/hardware/hpux.py
Including module_utils file ansible/module_utils/facts/hardware/hurd.py
Including module_utils file ansible/module_utils/facts/hardware/linux.py
Including module_utils file ansible/module_utils/facts/hardware/netbsd.py
Including module_utils file ansible/module_utils/facts/hardware/openbsd.py
Including module_utils file ansible/module_utils/facts/hardware/sunos.py
Including module_utils file ansible/module_utils/facts/namespace.py
Including module_utils file ansible/module_utils/facts/network/aix.py
Including module_utils file ansible/module_utils/facts/network/__init__.py
Including module_utils file ansible/module_utils/facts/network/base.py
Including module_utils file ansible/module_utils/facts/network/darwin.py
Including module_utils file ansible/module_utils/facts/network/dragonfly.py
Including module_utils file ansible/module_utils/facts/network/fc_wwn.py
Including module_utils file ansible/module_utils/facts/network/freebsd.py
Including module_utils file ansible/module_utils/facts/network/generic_bsd.py
Including module_utils file ansible/module_utils/facts/network/hpux.py
Including module_utils file ansible/module_utils/facts/network/hurd.py
Including module_utils file ansible/module_utils/facts/network/iscsi.py
Including module_utils file ansible/module_utils/facts/network/linux.py
Including module_utils file ansible/module_utils/facts/network/netbsd.py
Including module_utils file ansible/module_utils/facts/network/nvme.py
Including module_utils file ansible/module_utils/facts/network/openbsd.py
Including module_utils file ansible/module_utils/facts/network/sunos.py
Including module_utils file ansible/module_utils/facts/other/facter.py
Including module_utils file ansible/module_utils/facts/other/__init__.py
Including module_utils file ansible/module_utils/facts/other/ohai.py
Including module_utils file ansible/module_utils/facts/sysctl.py
Including module_utils file ansible/module_utils/facts/system/apparmor.py
Including module_utils file ansible/module_utils/facts/system/__init__.py
Including module_utils file ansible/module_utils/facts/system/caps.py
Including module_utils file ansible/module_utils/facts/system/chroot.py
Including module_utils file ansible/module_utils/facts/system/cmdline.py
Including module_utils file ansible/module_utils/facts/system/date_time.py
Including module_utils file ansible/module_utils/facts/system/distribution.py
Including module_utils file ansible/module_utils/facts/system/dns.py
Including module_utils file ansible/module_utils/facts/system/env.py
Including module_utils file ansible/module_utils/facts/system/fips.py
Including module_utils file ansible/module_utils/facts/system/local.py
Including module_utils file ansible/module_utils/facts/system/lsb.py
Including module_utils file ansible/module_utils/facts/system/pkg_mgr.py
Including module_utils file ansible/module_utils/facts/system/platform.py
Including module_utils file ansible/module_utils/facts/system/python.py
Including module_utils file ansible/module_utils/facts/system/selinux.py
Including module_utils file ansible/module_utils/facts/system/service_mgr.py
Including module_utils file ansible/module_utils/compat/version.py
Including module_utils file ansible/module_utils/facts/system/ssh_pub_keys.py
Including module_utils file ansible/module_utils/facts/system/user.py
Including module_utils file ansible/module_utils/facts/timeout.py
Including module_utils file ansible/module_utils/facts/utils.py
Including module_utils file ansible/module_utils/facts/virtual/base.py
Including module_utils file ansible/module_utils/facts/virtual/__init__.py
Including module_utils file ansible/module_utils/facts/virtual/dragonfly.py
Including module_utils file ansible/module_utils/facts/virtual/freebsd.py
Including module_utils file ansible/module_utils/facts/virtual/hpux.py
Including module_utils file ansible/module_utils/facts/virtual/linux.py
Including module_utils file ansible/module_utils/facts/virtual/netbsd.py
Including module_utils file ansible/module_utils/facts/virtual/openbsd.py
Including module_utils file ansible/module_utils/facts/virtual/sunos.py
Including module_utils file ansible/module_utils/facts/virtual/sysctl.py
Including module_utils file ansible/module_utils/parsing/convert_bool.py
Including module_utils file ansible/module_utils/parsing/__init__.py
Including module_utils file ansible/module_utils/pycompat24.py
Including module_utils file ansible/module_utils/six/__init__.py
Using module file /Users/josh/Code/server/hapy/venv/lib/python3.8/site-packages/ansible/modules/setup.py
<52.40.31.173> PUT /Users/josh/.ansible/tmp/ansible-local-55603n_9_500/tmpu3x5r1hf TO /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097/AnsiballZ_setup.py
<52.40.31.173> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ControlPath=/tmp/ansible-ssh-%h-%p-%r)(-o)(ForwardAgent=yes)
<52.40.31.173> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<52.40.31.173> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<52.40.31.173> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<52.40.31.173> SSH: Set ssh_common_args: ()
<52.40.31.173> SSH: Set sftp_extra_args: ()
<52.40.31.173> SSH: EXEC sftp -b - -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 '[52.40.31.173]'
<52.40.31.173> (0, b'sftp> put /Users/josh/.ansible/tmp/ansible-local-55603n_9_500/tmpu3x5r1hf /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097/AnsiballZ_setup.py\n', b'OpenSSH_8.6p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/josh/.ssh/config\r\ndebug1: /Users/josh/.ssh/config line 1: Applying options for *\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 52.40.31.173 is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/Users/josh/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/Users/josh/.ssh/known_hosts2\'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 5546\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "posix-rename@openssh.com" revision 1\r\ndebug2: Server supports extension "statvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "fstatvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "hardlink@openssh.com" revision 1\r\ndebug2: Server supports extension "fsync@openssh.com" revision 1\r\ndebug3: Sent message fd 3 T:16 I:1\r\ndebug3: SSH_FXP_REALPATH . -> /home/josh size 0\r\ndebug3: Looking up /Users/josh/.ansible/tmp/ansible-local-55603n_9_500/tmpu3x5r1hf\r\ndebug3: Sent message fd 3 T:17 I:2\r\ndebug3: Received stat reply T:101 I:2\r\ndebug1: Couldn\'t stat remote file: No such file or directory\r\ndebug3: Sent message SSH2_FXP_OPEN I:3 P:/home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097/AnsiballZ_setup.py\r\ndebug3: Sent message SSH2_FXP_WRITE I:4 O:0 S:32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 4 32768 bytes at 0\r\ndebug3: Sent message SSH2_FXP_WRITE I:5 O:32768 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:65536 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:7 O:98304 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:8 O:131072 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:9 O:163840 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:10 O:196608 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:11 O:229376 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:12 O:262144 S:32768\r\ndebug3: Sent message SSH2_FXP_WRITE I:13 O:294912 S:478\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 5 32768 bytes at 32768\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 32768 bytes at 65536\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 7 32768 bytes at 98304\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 8 32768 bytes at 131072\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 9 32768 bytes at 163840\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 10 32768 bytes at 196608\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 11 32768 bytes at 229376\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 12 32768 bytes at 262144\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 13 478 bytes at 294912\r\ndebug3: Sent message SSH2_FXP_CLOSE I:4\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<52.40.31.173> ESTABLISH SSH CONNECTION FOR USER: None
<52.40.31.173> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ControlPath=/tmp/ansible-ssh-%h-%p-%r)(-o)(ForwardAgent=yes)
<52.40.31.173> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<52.40.31.173> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<52.40.31.173> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<52.40.31.173> SSH: Set ssh_common_args: ()
<52.40.31.173> SSH: Set ssh_extra_args: ()
<52.40.31.173> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 52.40.31.173 '/bin/sh -c '"'"'chmod u+x /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097/ /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097/AnsiballZ_setup.py && sleep 0'"'"''
<52.40.31.173> (0, b'', b"OpenSSH_8.6p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/josh/.ssh/config\r\ndebug1: /Users/josh/.ssh/config line 1: Applying options for *\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 52.40.31.173 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/josh/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/josh/.ssh/known_hosts2'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 5546\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<52.40.31.173> ESTABLISH SSH CONNECTION FOR USER: None
<52.40.31.173> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ControlPath=/tmp/ansible-ssh-%h-%p-%r)(-o)(ForwardAgent=yes)
<52.40.31.173> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<52.40.31.173> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<52.40.31.173> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<52.40.31.173> SSH: Set ssh_common_args: ()
<52.40.31.173> SSH: Set ssh_extra_args: ()
<52.40.31.173> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -tt 52.40.31.173 '/bin/sh -c '"'"'python /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097/AnsiballZ_setup.py && sleep 0'"'"''
<52.40.31.173> (0, b'\r\n{"invocation": {"module_args": {"filter": [], "gather_subset": ["all"], "fact_path": "/etc/ansible/facts.d", "gather_timeout": 10}}, "ansible_facts": {"ansible_fibre_channel_wwn": [], "module_setup": true, "ansible_distribution_version": "2", "ansible_board_name": "NA", "ansible_distribution_file_variety": "Amazon", "ansible_env": {"H_HOME": "/home/ha", "LC_CTYPE": "en_US.UTF-8", "LESSOPEN": "||/usr/bin/lesspipe.sh %s", "SSH_CLIENT": "24.5.49.183 49359 22", "LOGNAME": "josh", "USER": "josh", "PATH": "/usr/local/bin:/usr/bin", "HOME": "/home/josh", "LANG": "en_US.UTF-8", "TERM": "xterm-256color", "SHELL": "/bin/bash", "SHLVL": "2", "H_LOGS": "/home/ha/logs", "XDG_RUNTIME_DIR": "/run/user/1005", "SSH_AUTH_SOCK": "/tmp/ssh-lkmBz2ozdM/agent.11752", "XDG_SESSION_ID": "208", "_": "/usr/bin/python", "SSH_CONNECTION": "24.5.49.183 49359 172.31.23.199 22", "SSH_TTY": "/dev/pts/0", "H_ENV": "prod", "PWD": "/home/josh", "H_DATA": "/home/ha/data", "MAIL": "/var/mail/josh", "LS_COLORS": "rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:"}, "ansible_userspace_bits": "64", "ansible_architecture": "x86_64", "ansible_default_ipv4": {"macaddress": "02:de:e6:19:0f:33", "network": "172.31.16.0", "netmask": "255.255.240.0", "mtu": 9001, "broadcast": "172.31.31.255", "alias": "eth0", "prefix": "20", "address": "172.31.23.199", "interface": "eth0", "type": "ether", "gateway": "172.31.16.1"}, "ansible_swapfree_mb": 0, "ansible_default_ipv6": {}, "ansible_cmdline": {"LANG": "en_US.UTF-8", "BOOT_IMAGE": "/boot/vmlinuz-4.14.291-218.527.amzn2.x86_64", "rd.emergency": "poweroff", "net.ifnames": "0", "KEYTABLE": "us", "biosdevname": "0", "rd.shell": "0", "console": "ttyS0,115200n8", "ro": true, "root": "UUID=e75a1891-3463-448b-8f59-5e3353af90ba", "nvme_core.io_timeout": "4294967295"}, "ansible_ssh_host_key_ed25519_public_keytype": "ssh-ed25519", "ansible_machine_id": "ec214dee1f497e5a0c3f972da49e4684", "ansible_userspace_architecture": "x86_64", "ansible_product_uuid": "NA", "ansible_pkg_mgr": "yum", "ansible_distribution": "Amazon", "ansible_iscsi_iqn": "", "ansible_all_ipv6_addresses": ["fe80::de:e6ff:fe19:f33"], "ansible_uptime_seconds": 86915, "ansible_kernel": "4.14.291-218.527.amzn2.x86_64", "ansible_system_capabilities_enforced": "True", "ansible_python": {"executable": "/usr/bin/python", "version": {"micro": 18, "major": 2, "releaselevel": "final", "serial": 0, "minor": 7}, "type": "CPython", "has_sslcontext": true, "version_info": [2, 7, 18, "final", 0]}, "ansible_is_chroot": true, "gather_subset": ["all"], "ansible_board_asset_tag": "NA", "ansible_user_shell": "/bin/bash", "ansible_product_serial": "NA", "ansible_form_factor": "Other", "ansible_distribution_file_parsed": true, "ansible_fips": false, "ansible_user_id": "josh", "ansible_selinux_python_present": true, "ansible_board_serial": "NA", "ansible_kernel_version": "#1 SMP Fri Aug 26 09:54:31 UTC 2022", "ansible_local": {}, "ansible_processor_vcpus": 2, "ansible_processor": ["0", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz", "1", "GenuineIntel", "Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz"], "ansible_ssh_host_key_ecdsa_public": "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM9cblzKUYU9jeh7aOs2Zwqys7sw4PJPwjI4xGrpa/FN/OGsfvqie2Rqx89zoyaMN4ym4IYyNuioJlqTeRs9JhU=", "ansible_user_gid": 1005, "ansible_system_vendor": "Xen", "ansible_swaptotal_mb": 0, "ansible_distribution_major_version": "2", "ansible_real_group_id": 1005, "ansible_lsb": {}, "ansible_machine": "x86_64", "ansible_ssh_host_key_rsa_public": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDKQRebsMqwuU234rHkSMKyvJE3N56nKJJ8xlOVdKHqkK1UoVHs/TgczMunAIIXPorec7E4Wx/P572jgV17s9qDZy/gU4nnppdJ3gYsVZGFgtPExy3SUsMvV3kqdWTF0ME51a6MlWWeBCefxmbt0KIcN/EBEHf4YLA0VNvvY1PIjlR8DmwZJpUoj6qyFRkYUmucYdnOXQnZTEv8FV3bpKlRiXKP589uHh1xR4M2ndH+depO9Zhsjy6gY49MEePLHdIR3r9UZgkrvw7sEVJOrSvAaujJHthNIR9chYvs/uSJK7VMMTSn11tCaWH+sEDSQMx1cfjpG2t7lufcFmwzNwPb", "ansible_user_gecos": "", "ansible_processor_threads_per_core": 1, "ansible_eth0": {"macaddress": "02:de:e6:19:0f:33", "features": {"tx_checksum_ipv4": "on [fixed]", "generic_receive_offload": "on", "tx_checksum_ipv6": "on", "tx_scatter_gather_fraglist": "off [fixed]", "rx_all": "off [fixed]", "highdma": "off [fixed]", "rx_fcs": "off [fixed]", "tx_lockless": "off [fixed]", "tx_tcp_ecn_segmentation": "off [fixed]", "rx_udp_tunnel_port_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "tx_gso_robust": "on [fixed]", "tx_tcp_mangleid_segmentation": "off", "tx_ipxip4_segmentation": "off [fixed]", "tx_checksumming": "on", "vlan_challenged": "off [fixed]", "loopback": "off [fixed]", "tx_ipxip6_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "scatter_gather": "on", "tx_checksum_sctp": "off [fixed]", "tx_vlan_stag_hw_insert": "off [fixed]", "rx_vlan_stag_hw_parse": "off [fixed]", "tx_gso_partial": "off [fixed]", "rx_vlan_stag_filter": "off [fixed]", "large_receive_offload": "off [fixed]", "tx_scatter_gather": "on", "rx_checksumming": "on [fixed]", "tx_tcp_segmentation": "on", "netns_local": "off [fixed]", "tx_esp_segmentation": "off [fixed]", "generic_segmentation_offload": "on", "tx_udp_tnl_segmentation": "off [fixed]", "tcp_segmentation_offload": "on", "l2_fwd_offload": "off [fixed]", "rx_vlan_offload": "off [fixed]", "ntuple_filters": "off [fixed]", "tx_gre_csum_segmentation": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "tx_tcp6_segmentation": "on", "tx_udp_tnl_csum_segmentation": "off [fixed]", "udp_fragmentation_offload": "off", "tx_sctp_segmentation": "off [fixed]", "tx_checksum_fcoe_crc": "off [fixed]", "hw_tc_offload": "off [fixed]", "tx_checksum_ip_generic": "off [fixed]", "tx_fcoe_segmentation": "off [fixed]", "tx_nocache_copy": "off", "rx_vlan_filter": "off [fixed]", "tx_vlan_offload": "off [fixed]", "receive_hashing": "off [fixed]", "tx_gre_segmentation": "off [fixed]"}, "pciid": "vif-0", "module": "xen_netfront", "mtu": 9001, "device": "eth0", "promisc": false, "timestamping": ["rx_software", "software"], "ipv4": {"broadcast": "172.31.31.255", "netmask": "255.255.240.0", "address": "172.31.23.199", "prefix": "20", "network": "172.31.16.0"}, "ipv6": [{"scope": "link", "prefix": "64", "address": "fe80::de:e6ff:fe19:f33"}], "active": true, "type": "ether", "hw_timestamp_filters": []}, "ansible_system": "Linux", "ansible_processor_nproc": 2, "ansible_bios_vendor": "Xen", "ansible_all_ipv4_addresses": ["172.31.23.199"], "ansible_python_version": "2.7.18", "ansible_virtualization_tech_guest": ["xen"], "ansible_product_version": "4.2.amazon", "ansible_service_mgr": "systemd", "ansible_memory_mb": {"real": {"total": 3941, "used": 1358, "free": 2583}, "swap": {"cached": 0, "total": 0, "free": 0, "used": 0}, "nocache": {"used": 301, "free": 3640}}, "ansible_user_dir": "/home/josh", "ansible_hostnqn": "", "ansible_real_user_id": 1005, "ansible_virtualization_role": "guest", "ansible_dns": {"nameservers": ["172.31.0.2"], "search": ["us-west-2.compute.internal", "ha"], "options": {"attempts": "5", "timeout": "2"}}, "ansible_effective_group_id": 1005, "ansible_lo": {"features": {"tx_checksum_ipv4": "off [fixed]", "generic_receive_offload": "on", "tx_checksum_ipv6": "off [fixed]", "tx_scatter_gather_fraglist": "on [fixed]", "rx_all": "off [fixed]", "highdma": "on [fixed]", "rx_fcs": "off [fixed]", "tx_lockless": "on [fixed]", "tx_tcp_ecn_segmentation": "on", "rx_udp_tunnel_port_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "tx_gso_robust": "off [fixed]", "tx_tcp_mangleid_segmentation": "on", "tx_ipxip4_segmentation": "off [fixed]", "tx_checksumming": "on", "vlan_challenged": "on [fixed]", "loopback": "on [fixed]", "tx_ipxip6_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "scatter_gather": "on", "tx_checksum_sctp": "on [fixed]", "tx_vlan_stag_hw_insert": "off [fixed]", "rx_vlan_stag_hw_parse": "off [fixed]", "tx_gso_partial": "off [fixed]", "rx_vlan_stag_filter": "off [fixed]", "large_receive_offload": "off [fixed]", "tx_scatter_gather": "on [fixed]", "rx_checksumming": "on [fixed]", "tx_tcp_segmentation": "on", "netns_local": "on [fixed]", "tx_esp_segmentation": "off [fixed]", "generic_segmentation_offload": "on", "tx_udp_tnl_segmentation": "off [fixed]", "tcp_segmentation_offload": "on", "l2_fwd_offload": "off [fixed]", "rx_vlan_offload": "off [fixed]", "ntuple_filters": "off [fixed]", "tx_gre_csum_segmentation": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "tx_tcp6_segmentation": "on", "tx_udp_tnl_csum_segmentation": "off [fixed]", "udp_fragmentation_offload": "off", "tx_sctp_segmentation": "on", "tx_checksum_fcoe_crc": "off [fixed]", "hw_tc_offload": "off [fixed]", "tx_checksum_ip_generic": "on [fixed]", "tx_fcoe_segmentation": "off [fixed]", "tx_nocache_copy": "off [fixed]", "rx_vlan_filter": "off [fixed]", "tx_vlan_offload": "off [fixed]", "receive_hashing": "off [fixed]", "tx_gre_segmentation": "off [fixed]"}, "hw_timestamp_filters": [], "mtu": 65536, "device": "lo", "promisc": false, "timestamping": ["tx_software", "rx_software", "software"], "ipv4": {"broadcast": "", "netmask": "255.0.0.0", "address": "127.0.0.1", "prefix": "8", "network": "127.0.0.0"}, "ipv6": [{"scope": "host", "prefix": "128", "address": "::1"}], "active": true, "type": "loopback"}, "ansible_memtotal_mb": 3941, "ansible_device_links": {"masters": {}, "labels": {"xvda1": ["\\\\x2f"]}, "ids": {}, "uuids": {"xvda1": ["e75a1891-3463-448b-8f59-5e3353af90ba"]}}, "ansible_apparmor": {"status": "disabled"}, "ansible_board_vendor": "NA", "ansible_bios_date": "08/24/2006", "ansible_memfree_mb": 2583, "ansible_board_version": "NA", "ansible_chassis_vendor": "Xen", "ansible_virtualization_tech_host": [], "ansible_processor_count": 1, "ansible_hostname": "a-test-0", "ansible_interfaces": ["lo", "eth0"], "ansible_selinux": {"status": "disabled"}, "ansible_fqdn": "a-test-0.ha", "ansible_mounts": [{"block_used": 1049471, "uuid": "e75a1891-3463-448b-8f59-5e3353af90ba", "size_total": 107361579008, "block_total": 26211323, "mount": "/", "block_available": 25161852, "size_available": 103062945792, "fstype": "xfs", "inode_total": 52427712, "options": "rw,noatime,attr2,inode64,noquota", "device": "/dev/xvda1", "inode_used": 98890, "block_size": 4096, "inode_available": 52328822}], "ansible_ssh_host_key_rsa_public_keytype": "ssh-rsa", "ansible_nodename": "a-test-0.ha", "ansible_distribution_minor_version": "NA", "ansible_domain": "ha", "ansible_distribution_file_path": "/etc/os-release", "ansible_virtualization_type": "xen", "ansible_ssh_host_key_ed25519_public": "AAAAC3NzaC1lZDI1NTE5AAAAIGsvEEAfU3betI6eK7R9CGst3DaGpGDOoBlaR00H61ay", "ansible_processor_cores": 2, "ansible_chassis_serial": "NA", "ansible_bios_version": "4.2.amazon", "ansible_date_time": {"weekday_number": "4", "iso8601_basic_short": "20221013T224821", "tz": "UTC", "weeknumber": "41", "hour": "22", "year": "2022", "epoch_int": "1665701301", "tz_offset": "+0000", "month": "10", "epoch": "1665701301", "iso8601_micro": "2022-10-13T22:48:21.021862Z", "weekday": "Thursday", "minute": "48", "time": "22:48:21", "date": "2022-10-13", "iso8601": "2022-10-13T22:48:21Z", "tz_dst": "UTC", "day": "13", "iso8601_basic": "20221013T224821021862", "second": "21"}, "ansible_distribution_release": "NA", "ansible_os_family": "RedHat", "ansible_effective_user_id": 1005, "ansible_ssh_host_key_ecdsa_public_keytype": "ecdsa-sha2-nistp256", "ansible_product_name": "HVM domU", "ansible_devices": {"xvda": {"scheduler_mode": "noop", "rotational": "0", "vendor": null, "sectors": "209715200", "links": {"masters": [], "labels": [], "ids": [], "uuids": []}, "sas_device_handle": null, "sas_address": null, "virtual": 1, "host": "", "sectorsize": "512", "removable": "0", "support_discard": "0", "model": null, "partitions": {"xvda1": {"sectorsize": 512, "uuid": "e75a1891-3463-448b-8f59-5e3353af90ba", "links": {"masters": [], "labels": ["\\\\x2f"], "ids": [], "uuids": ["e75a1891-3463-448b-8f59-5e3353af90ba"]}, "sectors": "209711071", "start": "4096", "holders": [], "size": "100.00 GB"}}, "holders": [], "size": "100.00 GB"}}, "ansible_user_uid": 1005, "ansible_proc_cmdline": {"LANG": "en_US.UTF-8", "BOOT_IMAGE": "/boot/vmlinuz-4.14.291-218.527.amzn2.x86_64", "rd.emergency": "poweroff", "net.ifnames": "0", "KEYTABLE": "us", "biosdevname": "0", "rd.shell": "0", "console": ["tty0", "ttyS0,115200n8"], "ro": true, "root": "UUID=e75a1891-3463-448b-8f59-5e3353af90ba", "nvme_core.io_timeout": "4294967295"}, "ansible_chassis_version": "NA", "ansible_system_capabilities": [""], "ansible_chassis_asset_tag": "NA"}}\r\n', b"OpenSSH_8.6p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/josh/.ssh/config\r\ndebug1: /Users/josh/.ssh/config line 1: Applying options for *\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 52.40.31.173 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/josh/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/josh/.ssh/known_hosts2'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 5546\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to 52.40.31.173 closed.\r\n")
<52.40.31.173> ESTABLISH SSH CONNECTION FOR USER: None
<52.40.31.173> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=60s)(-o)(ControlPath=/tmp/ansible-ssh-%h-%p-%r)(-o)(ForwardAgent=yes)
<52.40.31.173> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<52.40.31.173> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<52.40.31.173> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<52.40.31.173> SSH: Set ssh_common_args: ()
<52.40.31.173> SSH: Set ssh_extra_args: ()
<52.40.31.173> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r -o ForwardAgent=yes -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 52.40.31.173 '/bin/sh -c '"'"'rm -f -r /home/josh/.ansible/tmp/ansible-tmp-1665701298.5003798-5566-235954551572097/ > /dev/null 2>&1 && sleep 0'"'"''
<52.40.31.173> (0, b'', b"OpenSSH_8.6p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/josh/.ssh/config\r\ndebug1: /Users/josh/.ssh/config line 1: Applying options for *\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 52.40.31.173 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/josh/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/josh/.ssh/known_hosts2'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 5546\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")

TASK [Gathering Facts] ****************************************************************************************************************************************************************************
task path: /Users/josh/Code/server/ansible/h.yml:1
ok: [a-test-0@52.40.31.173]
META: ran handlers
ERROR! A worker was found in a dead state

This also triggered my OS to detect a crash in Python. Some info from the crash report (I can paste more info from the Apple crash report if needed, but I thought this encompassed anything relevant):

Exception Type:        EXC_GUARD (SIGKILL)
Exception Codes:       GUARD_TYPE_MACH_PORT
Exception Codes:       0x0000000000001903, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace GUARD, Code 2305845208236955907 

Application Specific Information:
*** multi-threaded process forked ***
crashed on child side of fork pre-exec

Thread 0 Crashed::  Dispatch queue: SCDynamicStore object
0   libsystem_kernel.dylib                 0x1bf65a820 _kernelrpc_mach_port_deallocate_trap + 8
1   libsystem_kernel.dylib                 0x1bf65bde8 mach_port_deallocate + 28
2   SystemConfiguration                    0x1c02f6820 ____SCDynamicStoreCheckRetryAndHandleError_block_invoke + 68
3   libdispatch.dylib                      0x1bf4d41b4 _dispatch_client_callout + 20
4   libdispatch.dylib                      0x1bf4e3414 _dispatch_lane_barrier_sync_invoke_and_complete + 56
5   SystemConfiguration                    0x1c02d9cc8 __SCDynamicStoreCheckRetryAndHandleError + 180
6   SystemConfiguration                    0x1c02d9930 SCDynamicStoreCopyValue + 288
7   SystemConfiguration                    0x1c02ddcfc SCDynamicStoreCopyProxiesWithOptions + 580
8   _scproxy.cpython-38-darwin.so          0x10373f75c get_proxy_settings + 44
9   Python3                                0x1011c4fac cfunction_vectorcall_NOARGS + 240
10  Python3                                0x10125236c call_function + 288
11  Python3                                0x10124f79c _PyEval_EvalFrameDefault + 23576
12  Python3                                0x101185d3c function_code_fastcall + 120
13  Python3                                0x10125236c call_function + 288

Code of Conduct

ansibullbot commented 1 year ago

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

kennedyjosh commented 1 year ago

Just after posting this I had a thought and tried calling the ansible-playbook command directly from within the virtual env, skipping over python and cli script. The error still occurs.

kennedyjosh commented 1 year ago

!component=plugins/lookup/aws_secret.py

ansibullbot commented 1 year ago

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

gravesm commented 1 year ago

@kennedyjosh This is likely the same problem discussed in https://github.com/ansible/ansible/issues/76322. You can try setting OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES as suggested in that discussion.

kennedyjosh commented 1 year ago

This is likely the same problem discussed in https://github.com/ansible/ansible/issues/76322. You can try setting OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES as suggested in that discussion.

I noticed that issue when researching the problem. Unfortunately, that fix doesn't seem to work for me.

kennedyjosh commented 1 year ago

I was just able to solve the issue by deleting and re-installing the virtual environment. I'm still not sure what the underlying cause was.

kennedyjosh commented 1 year ago

Closing this as it doesn't seem to be a problem with ansible