search

ansible-collections / community.aws

Ansible Collection for Community AWS
GNU General Public License v3.0
189 stars 398 forks source link

copy module crashes with empty content when using the SSM plugin #1686

Closed bencehornak-gls closed 1 year ago

bencehornak-gls commented 1 year ago

Summary

When I execute the builtin copy module with the content parameter set to empty string, I get the following error (taken from the -vvvv logs):

<i-XXXXXXXXd68> EXEC stdout line: chmod: cannot access ‘/tmp/.ansible-/ansible-tmp-1675251475.8058956-7383-181169058616495/source’: No such file or directory

Issue Type

Bug Report

Component Name

ssm_plugin

Ansible Version

$ ansible --version
ansible [core 2.14.1]
  config file = /XXXX/ansible.cfg
  configured module search path = ['/XXXX/library']
  ansible python module location = /home/bence/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/bence/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/bence/.local/bin/ansible
  python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
# /XXX/lib/python3.10/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    5.1.0  
ansible.netcommon             4.1.0  
ansible.posix                 1.4.0  
ansible.utils                 2.8.0  
ansible.windows               1.12.0 
arista.eos                    6.0.0  
awx.awx                       21.10.0
azure.azcollection            1.14.0 
check_point.mgmt              4.0.0  
chocolatey.chocolatey         1.3.1  
cisco.aci                     2.3.0  
cisco.asa                     4.0.0  
cisco.dnac                    6.6.1  
cisco.intersight              1.0.22 
cisco.ios                     4.0.0  
cisco.iosxr                   4.0.3  
cisco.ise                     2.5.9  
cisco.meraki                  2.13.0 
cisco.mso                     2.1.0  
cisco.nso                     1.0.3  
cisco.nxos                    4.0.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.3  
community.aws                 5.0.0  
community.azure               2.0.0  
community.ciscosmb            1.0.5  
community.crypto              2.9.0  
community.digitalocean        1.22.0 
community.dns                 2.4.2  
community.docker              3.3.1  
community.fortios             1.0.0  
community.general             6.1.0  
community.google              1.0.0  
community.grafana             1.5.3  
community.hashi_vault         4.0.0  
community.hrobot              1.6.0  
community.libvirt             1.2.0  
community.mongodb             1.4.2  
community.mysql               3.5.1  
community.network             5.0.0  
community.okd                 2.2.0  
community.postgresql          2.3.1  
community.proxysql            1.4.0  
community.rabbitmq            1.2.3  
community.routeros            2.5.0  
community.sap                 1.0.0  
community.sap_libs            1.4.0  
community.skydive             1.0.0  
community.sops                1.5.0  
community.vmware              3.2.0  
community.windows             1.11.1 
community.zabbix              1.9.0  
containers.podman             1.10.1 
cyberark.conjur               1.2.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      2.0.0  
dellemc.openmanage            6.3.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.21.0 
fortinet.fortimanager         2.1.7  
fortinet.fortios              2.2.1  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
grafana.grafana               1.1.0  
hetzner.hcloud                1.9.0  
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       1.10.0 
infinidat.infinibox           1.3.12 
infoblox.nios_modules         1.4.1  
inspur.ispim                  1.2.0  
inspur.sm                     2.3.0  
junipernetworks.junos         4.1.0  
kubernetes.core               2.3.2  
lowlydba.sqlserver            1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.21.0
netapp.elementsw              21.7.0 
netapp.ontap                  22.0.1 
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.1  
netbox.netbox                 3.9.0  
ngine_io.cloudstack           2.3.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.10.0 
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.4.1  
purestorage.flasharray        1.15.0 
purestorage.flashblade        1.10.0 
purestorage.fusion            1.2.0  
sensu.sensu_go                1.13.1 
splunk.es                     2.1.0  
t_systems_mms.icinga_director 1.31.4 
theforeman.foreman            3.7.0  
vmware.vmware_rest            2.2.0  
vultr.cloud                   1.3.1  
vyos.vyos                     4.0.0  
wti.remote                    1.0.4

AWS SDK versions

$ pip show boto boto3 botocore
pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.26.52
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/bence/gls/devops/ansible-parcelos/tests/.env/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.29.52
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/bence/gls/devops/ansible-parcelos/tests/.env/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
ANY_ERRORS_FATAL(/XXXXXXXXXXXXXX/ansible.cfg) = True
CALLBACKS_ENABLED(/XXXXXXXXXXXXXX/ansible.cfg) = ['profile_tasks']
CONFIG_FILE() = /XXXXXXXXXXXXXX/ansible.cfg
DEFAULT_FORKS(/XXXXXXXXXXXXXX/ansible.cfg) = 25
DEFAULT_MODULE_PATH(/XXXXXXXXXXXXXX/ansible.cfg) = ['/XXXXXXXXXXXXXX/library']
DEFAULT_ROLES_PATH(/XXXXXXXXXXXXXX/ansible.cfg) = ['/XXXXXXXXXXXXXX/roles']
DEFAULT_TIMEOUT(/XXXXXXXXXXXXXX/ansible.cfg) = 20
DISPLAY_SKIPPED_HOSTS(/XXXXXXXXXXXXXX/ansible.cfg) = False
HOST_KEY_CHECKING(/XXXXXXXXXXXXXX/ansible.cfg) = False
MAX_FILE_SIZE_FOR_DIFF(/XXXXXXXXXXXXXX/ansible.cfg) = 1044480

OS / Environment

Windows 10 with WSL (Ubuntu 20)

Steps to Reproduce

Use the built-in copy module with the content set to empty string.

ansible -i inventories/aws_ec2.yml -m copy -a "dest=/tmp/test content=" target_host -vvv

Expected Results

The temporary source file should be an empty file (e.g. /tmp/.ansible-/ansible-tmp-1675251475.8058956-7383-181169058616495/source)

Actual Results

The temporary source file is not present, so the Ansible step crashes.

Output (sensitive parts are masked):

ansible -i inventories/aws_ec2.yml -m copy -a "dest=/tmp/test content=" target_host -vvv

...
redirecting (type: inventory) ansible.builtin.aws_ec2 to amazon.aws.aws_ec2
Using inventory plugin 'ansible_collections.amazon.aws.plugins.inventory.aws_ec2' to process inventory source '/XXXXXXXXXXXXXXXX/inventories/aws_ec2.yml'
Parsed /XXXXXXXXXXXXXXXX/inventories/aws_ec2.yml inventory source with auto plugin
redirecting (type: callback) ansible.builtin.profile_tasks to ansible.posix.profile_tasks
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.
redirecting (type: connection) ansible.builtin.aws_ssm to community.aws.aws_ssm
<i-XXXXXXXXXXXXXXX68> ESTABLISH SSM CONNECTION TO: i-XXXXXXXXXXXXXXX68
<i-XXXXXXXXXXXXXXX68> SSM CONNECTION ID: XXXXXXXXXXXXXXXXXXX
<i-XXXXXXXXXXXXXXX68> EXEC ( umask 77 && mkdir -p "` echo /tmp/.ansible-${USER}/ `"&& mkdir "` echo /tmp/.ansible-${USER}/ansible-tmp-1675252624.7794013-9014-861916818482 `" && echo ansible-tmp-1675252624.7794013-9014-861916818482="` echo /tmp/.ansible-${USER}/ansible-tmp-1675252624.7794013-9014-861916818482 `" )
<i-XXXXXXXXXXXXXXX68> (0, 'ansible-tmp-1675252624.7794013-9014-861916818482=/tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482\r\r', '')
<i-XXXXXXXXXXXXXXX68> Attempting python interpreter discovery
<i-XXXXXXXXXXXXXXX68> EXEC echo PLATFORM; uname; echo FOUND; command -v 'python3.11'; command -v 'python3.10'; command -v 'python3.9'; command -v 'python3.8'; command -v 'python3.7'; command -v 'python3.6'; command -v 'python3.5'; command -v '/usr/bin/python3'; command -v '/usr/libexec/platform-python'; command -v 'python2.7'; command -v '/usr/bin/python'; command -v 'python'; echo ENDFOUND
<i-XXXXXXXXXXXXXXX68> (0, 'PLATFORM\r\r\nLinux\r\r\nFOUND\r\r\n/usr/bin/python3.6\r\r\n/usr/bin/python3\r\r\n/usr/libexec/platform-python\r\r\n/usr/bin/python2.7\r\r\n/usr/bin/python\r\r\n/usr/bin/python\r\r\nENDFOUND\r\r', '')
<i-XXXXXXXXXXXXXXX68> Python interpreter discovery fallback (pipelining support required for extended interpreter discovery)
Using module file /home/bence/.local/lib/python3.10/site-packages/ansible/modules/stat.py
<i-XXXXXXXXXXXXXXX68> PUT /home/bence/.ansible/tmp/ansible-local-8998ncfzil01/tmpg1ordya2 TO /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/AnsiballZ_stat.py
<i-XXXXXXXXXXXXXXX68> EXEC curl 'https://XXXXXXXXXXXXXXXXXXXXXXX.s3.amazonaws.com/i-XXXXXXXXXXXXXXX68//tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/AnsiballZ_stat.py' -o '/tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/AnsiballZ_stat.py'
<i-XXXXXXXXXXXXXXX68> (0, '  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\r\r\n                                 Dload  Upload   Total   Spent    Left  Speed\r\r\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100  129k  100  129k    0     0   542k      0 --:--:-- --:--:-- --:--:--  545k\r\r', '')
<i-XXXXXXXXXXXXXXX68> (0, '  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\r\r\n                                 Dload  Upload   Total   Spent    Left  Speed\r\r\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100  129k  100  129k    0     0   542k      0 --:--:-- --:--:-- --:--:--  545k\r\r', '')
<i-XXXXXXXXXXXXXXX68> EXEC chmod u+x /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/ /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/AnsiballZ_stat.py
<i-XXXXXXXXXXXXXXX68> (0, '\r', '')
<i-XXXXXXXXXXXXXXX68> EXEC /usr/bin/python3.6 /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/AnsiballZ_stat.py
<i-XXXXXXXXXXXXXXX68> (0, '\r\r\n{"changed": false, "stat": {"exists": false}, "invocation": {"module_args": {"path": "/tmp/test", "follow": false, "get_checksum": true, "checksum_algorithm": "sha1", "get_md5": false, "get_mime": true, "get_attributes": true}}}\r\r', '')
<i-XXXXXXXXXXXXXXX68> PUT /home/bence/.ansible/tmp/ansible-local-8998ncfzil01/tmp3b0hfmbo TO /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/source
<i-XXXXXXXXXXXXXXX68> EXEC curl 'https://XXXXXXXXXXXXXXXXXXXXXXX.s3.amazonaws.com/i-XXXXXXXXXXXXXXX68//tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/source?XXXXXXXXXXX' -o '/tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/source'
<i-XXXXXXXXXXXXXXX68> (0, '  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\r\r\n                                 Dload  Upload   Total   Spent    Left  Speed\r\r\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r\r', '')
<i-XXXXXXXXXXXXXXX68> (0, '  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\r\r\n                                 Dload  Upload   Total   Spent    Left  Speed\r\r\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r\r', '')
<i-XXXXXXXXXXXXXXX68> EXEC chmod u+x /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/ /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/source
<i-XXXXXXXXXXXXXXX68> (1, 'chmod: cannot access ‘/tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/source’: No such file or directory\r\r', '')
<i-XXXXXXXXXXXXXXX68> EXEC rm -f -r /tmp/.ansible-/ansible-tmp-1675252624.7794013-9014-861916818482/ > /dev/null 2>&1
<i-XXXXXXXXXXXXXXX68> (0, '\r', '')
<i-XXXXXXXXXXXXXXX68> CLOSING SSM CONNECTION TO: i-XXXXXXXXXXXXXXX68
i-XXXXXXXXXXXXXXX68 | FAILED! => {
    "msg": "Failed to set execute bit on remote files (rc: 1, err: )"
}

Note: if you repeat the steps with a host with SSH connection, the step just completes without error.

Code of Conduct

tremble commented 1 year ago

@bencehornak-gls

Thanks for taking the time to open this issue. You're currently using release 5.0.0, we released 5.2.0 last week with a number of bugfixes, please could you try again with 5.2.0

bencehornak-gls commented 1 year ago

Unfortunately the issue persists after upgrading to 5.2.0

galaxy modules ansible-galaxy collection list: ```console # /home/bence/.ansible/collections/ansible_collections Collection Version ------------- ------- amazon.aws 5.2.0 community.aws 5.2.0 # /home/bence/.local/lib/python3.10/site-packages/ansible_collections Collection Version ----------------------------- ------- amazon.aws 5.1.0 ansible.netcommon 4.1.0 ansible.posix 1.4.0 ansible.utils 2.8.0 ansible.windows 1.12.0 arista.eos 6.0.0 awx.awx 21.10.0 azure.azcollection 1.14.0 check_point.mgmt 4.0.0 chocolatey.chocolatey 1.3.1 cisco.aci 2.3.0 cisco.asa 4.0.0 cisco.dnac 6.6.1 cisco.intersight 1.0.22 cisco.ios 4.0.0 cisco.iosxr 4.0.3 cisco.ise 2.5.9 cisco.meraki 2.13.0 cisco.mso 2.1.0 cisco.nso 1.0.3 cisco.nxos 4.0.1 cisco.ucs 1.8.0 cloud.common 2.1.2 cloudscale_ch.cloud 2.2.3 community.aws 5.0.0 community.azure 2.0.0 community.ciscosmb 1.0.5 community.crypto 2.9.0 community.digitalocean 1.22.0 community.dns 2.4.2 community.docker 3.3.1 community.fortios 1.0.0 community.general 6.1.0 community.google 1.0.0 community.grafana 1.5.3 community.hashi_vault 4.0.0 community.hrobot 1.6.0 community.libvirt 1.2.0 community.mongodb 1.4.2 community.mysql 3.5.1 community.network 5.0.0 community.okd 2.2.0 community.postgresql 2.3.1 community.proxysql 1.4.0 community.rabbitmq 1.2.3 community.routeros 2.5.0 community.sap 1.0.0 community.sap_libs 1.4.0 community.skydive 1.0.0 community.sops 1.5.0 community.vmware 3.2.0 community.windows 1.11.1 community.zabbix 1.9.0 containers.podman 1.10.1 cyberark.conjur 1.2.0 cyberark.pas 1.0.14 dellemc.enterprise_sonic 2.0.0 dellemc.openmanage 6.3.0 dellemc.os10 1.1.1 dellemc.os6 1.0.7 dellemc.os9 1.0.4 f5networks.f5_modules 1.21.0 fortinet.fortimanager 2.1.7 fortinet.fortios 2.2.1 frr.frr 2.0.0 gluster.gluster 1.0.2 google.cloud 1.0.2 grafana.grafana 1.1.0 hetzner.hcloud 1.9.0 hpe.nimble 1.1.4 ibm.qradar 2.1.0 ibm.spectrum_virtualize 1.10.0 infinidat.infinibox 1.3.12 infoblox.nios_modules 1.4.1 inspur.ispim 1.2.0 inspur.sm 2.3.0 junipernetworks.junos 4.1.0 kubernetes.core 2.3.2 lowlydba.sqlserver 1.2.1 mellanox.onyx 1.0.0 netapp.aws 21.7.0 netapp.azure 21.10.0 netapp.cloudmanager 21.21.0 netapp.elementsw 21.7.0 netapp.ontap 22.0.1 netapp.storagegrid 21.11.1 netapp.um_info 21.8.0 netapp_eseries.santricity 1.3.1 netbox.netbox 3.9.0 ngine_io.cloudstack 2.3.0 ngine_io.exoscale 1.0.0 ngine_io.vultr 1.1.2 openstack.cloud 1.10.0 openvswitch.openvswitch 2.1.0 ovirt.ovirt 2.4.1 purestorage.flasharray 1.15.0 purestorage.flashblade 1.10.0 purestorage.fusion 1.2.0 sensu.sensu_go 1.13.1 splunk.es 2.1.0 t_systems_mms.icinga_director 1.31.4 theforeman.foreman 3.7.0 vmware.vmware_rest 2.2.0 vultr.cloud 1.3.1 vyos.vyos 4.0.0 wti.remote 1.0.4 ```
bencehornak-gls commented 1 year ago

Forgot to mention, but my workaround was to use content: '\n'. It is fine for my use case, because extra new-lines are ignored in this file, but might not be an option for some use-cases.

tremble commented 1 year ago

That's helpful information. It looks like this is basically curl doing something weird: https://github.com/curl/curl/issues/183 I'm not sure what the appropriate fix would be :/

bencehornak-gls commented 1 year ago

Yes, this seems to be the consequence of curl's crappy design here. I would also expect curl to create an empty file in this case.

For the time-being I don't see any other ways than adding a touch command after the curl command to make sure that it exists (even if it is supposed to be empty).

tremble commented 1 year ago

If you're willing to open a Pull Request we can probably get this patched and fixed pretty quickly:

https://github.com/ansible-collections/community.aws/blob/main/plugins/connection/aws_ssm.py#L832 with an extra test in https://github.com/ansible-collections/community.aws/blob/main/tests/integration/targets/connection/test_connection.yml