search

ansible-collections / amazon.aws

Ansible Collection for Amazon AWS
GNU General Public License v3.0
309 stars 341 forks source link

Cannot run amazon.aws.s3_object on windows machine with ansible #1942

Closed johndevera closed 10 months ago

johndevera commented 10 months ago

Summary

I'm on my Mac calling ansible on a windows device. Other ansible moduleswork like normal, but when I get to calling the amazon.aws.s3_object, it fails

I run:

Fails with the attached error ansible_error

Issue Type

Bug Report

Component Name

amazon.aws.s3_object

Ansible Version

$ ansible --version

ansible [core 2.16.2]
  config file = None
  configured module search path = ['/Users/XXXX/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/XXXXX/YYYYY/.venv/lib/python3.11/site-packages/ansible
  ansible collection location = /Users/XXXXX/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/XXXX/YYYY/.venv/bin/ansible
  python version = 3.11.6 (main, Nov 15 2023, 13:57:49) [Clang 14.0.3 (clang-1403.0.22.14.1)] (/Users/XXXX/YYYY/.venv/bin/python)
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /Users/XXXX/.ansible/collections/ansible_collections
Collection                    Version
----------------------------- -------
community.windows             2.0.0  

# /Users/XXXX/YYYY/.venv/lib/python3.11/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    7.0.0  
ansible.netcommon             5.3.0  
ansible.posix                 1.5.4  
ansible.utils                 2.12.0 
ansible.windows               2.2.0  
arista.eos                    6.2.2  
awx.awx                       23.5.0 
azure.azcollection            1.19.0 
check_point.mgmt              5.1.1  
chocolatey.chocolatey         1.5.1  
cisco.aci                     2.8.0  
cisco.asa                     4.0.3  
cisco.dnac                    6.8.1  
cisco.intersight              2.0.3  
cisco.ios                     5.3.0  
cisco.iosxr                   6.1.1  
cisco.ise                     2.6.2  
cisco.meraki                  2.16.16
cisco.mso                     2.5.0  
cisco.nxos                    5.3.0  
cisco.ucs                     1.10.0 
cloud.common                  2.1.4  
cloudscale_ch.cloud           2.3.1  
community.aws                 7.0.0  
community.azure               2.0.0  
community.ciscosmb            1.0.7  
community.crypto              2.16.1 
community.digitalocean        1.24.0 
community.dns                 2.6.4  
community.docker              3.4.11 
community.general             8.1.0  
community.grafana             1.6.1  
community.hashi_vault         6.0.0  
community.hrobot              1.8.2  
community.libvirt             1.3.0  
community.mongodb             1.6.3  
community.mysql               3.8.0  
community.network             5.0.2  
community.okd                 2.3.0  
community.postgresql          3.2.0  
community.proxysql            1.5.1  
community.rabbitmq            1.2.3  
community.routeros            2.11.0 
community.sap                 2.0.0  
community.sap_libs            1.4.1  
community.sops                1.6.7  
community.vmware              4.0.1  
community.windows             2.1.0  
community.zabbix              2.2.0  
containers.podman             1.11.0 
cyberark.conjur               1.2.2  
cyberark.pas                  1.0.23 
dellemc.enterprise_sonic      2.2.0  
dellemc.openmanage            8.5.0  
dellemc.powerflex             2.1.0  
dellemc.unity                 1.7.1  
f5networks.f5_modules         1.27.1 
fortinet.fortimanager         2.3.0  
fortinet.fortios              2.3.4  
frr.frr                       2.0.2  
gluster.gluster               1.0.2  
google.cloud                  1.3.0  
grafana.grafana               2.2.3  
hetzner.hcloud                2.4.1  
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       2.0.0  
ibm.storage_virtualize        2.1.0  
infinidat.infinibox           1.3.12 
infoblox.nios_modules         1.5.0  
inspur.ispim                  2.2.0  
inspur.sm                     2.3.0  
junipernetworks.junos         5.3.1  
kubernetes.core               2.4.0  
lowlydba.sqlserver            2.2.2  
microsoft.ad                  1.4.1  
netapp.aws                    21.7.1 
netapp.azure                  21.10.1
netapp.cloudmanager           21.22.1
netapp.elementsw              21.7.0 
netapp.ontap                  22.8.3 
netapp.storagegrid            21.11.1
netapp.um_info                21.8.1 
netapp_eseries.santricity     1.4.0  
netbox.netbox                 3.15.0 
ngine_io.cloudstack           2.3.0  
ngine_io.exoscale             1.1.0  
openstack.cloud               2.2.0  
openvswitch.openvswitch       2.1.1  
ovirt.ovirt                   3.2.0  
purestorage.flasharray        1.24.0 
purestorage.flashblade        1.14.0 
purestorage.fusion            1.6.0  
sensu.sensu_go                1.14.0 
splunk.es                     2.1.2  
t_systems_mms.icinga_director 2.0.1  
telekom_mms.icinga_director   1.35.0 
theforeman.foreman            3.15.0 
vmware.vmware_rest            2.3.1  
vultr.cloud                   1.10.1 
vyos.vyos                     4.1.0  
wti.remote                    1.0.5

AWS SDK versions

$ pip show boto boto3 botocore

WARNING: Package(s) not found: boto
Name: boto3
Version: 1.34.15
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/XXXX/YYYY/.venv/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.34.15
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/XXXX/YYYYY.venv/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: awscli, boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = None

OS / Environment

Running Ansible on macOS Ventura 13.4.1, Apple M2 Pro Inside Python 3.11.6 virtualenvironment with modules python = ">=3.10, <3.12.0" ansible = "9.1.0" ansible-core = "2.16.2" awscli = "1.32.15" boto3 = "1.34.15" pywinrm = "0.4.3"

Steps to Reproduce


- name: Install Windows software packages and programs
  hosts: itg_lab_group
  vars_files:
    - group_vars/all.yml
  vars:
    ansible_python_interpreter: .venv/bin/python3
  tasks:
    - name: Download file from S3
      amazon.aws.s3_object:
        aws_access_key: "{{ s3_vendor_access_key }}"
        aws_secret_key: "{{ s3_vendor_secret_key }}"
        bucket: "{{ s3_vendor_artifacts }}"
        object: "my_folder/my_file.txt"
        dest: "{{ download_directory }}"
        mode: get

Expected Results

The file from S3 called My_file.txt should be placed inside the directory, specified by "{{ download_directory }}"

Actual Results

fatal: [ABC123]: FAILED! => {
    "changed": false,
    "module_stderr": "Exception calling \"Create\" with \"1\" argument(s): \"At line:4 char:21\r\n+ def _ansiballz_main():\r\n+                     ~\r\nAn expression was expected after '('.\r\nAt line:8 char:19\r\n+         os.getcwd()\r\n+                   ~\r\nAn expression was expected after '('.\r\nAt line:20 char:27\r\n+     except (AttributeError, OSError):\r\n+                           ~\r\nMissing argument in parameter list.\r\nAt line:22 char:29\r\n+     excludes = set(('', '.', scriptdir))\r\n+                             ~\r\nMissing expression after ','.\r\nAt line:22 char:30\r\n+     excludes = set(('', '.', scriptdir))\r\n+                              ~~~~~~~~~\r\nUnexpected token 'scriptdir' in expression or statement.\r\nAt line:22 char:29\r\n+     excludes = set(('', '.', scriptdir))\r\n+                             ~\r\nMissing closing ')' in expression.\r\nAt line:22 char:39\r\n+     excludes = set(('', '.', scriptdir))\r\n+                                       ~\r\nUnexpected token ')' in expression or statement.\r\nAt line:22 char:40\r\n+     excludes = set(('', '.', scriptdir))\r\n+                                        ~\r\nUnexpected token ')' in expression or statement.\r\nAt line:29 char:7\r\n+     if sys.version_info < (3,):\r\n+       ~\r\nMissing '(' after 'if' in if statement.\r\nAt line:29 char:30\r\n+     if sys.version_info < (3,):\r\n+                              ~\r\nMissing expression after ','.\r\nNot all parse errors were reported.  Correct the reported errors and try again.\"\r\nAt line:10 char:1\r\n+ $exec_wrapper = [ScriptBlock]::Create($split_parts[0])\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException\r\n    + FullyQualifiedErrorId : ParseException\r\n \r\nThe expression after '&' in a pipeline element produced an object that was not valid. It \r\nmust result in a command name, a script block, or a CommandInfo object.\r\nAt line:11 char:2\r\n+ &$exec_wrapper\r\n+  ~~~~~~~~~~~~~\r\n    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException\r\n    + FullyQualifiedErrorId : BadExpression\r\n ",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
![ansible_error](https://github.com/ansible-collections/amazon.aws/assets/25378574/f447c940-7f13-45bc-8936-f12bf31d1e25)

Code of Conduct

tremble commented 10 months ago

Thanks for taking the time to open this issue. Unfortunately I believe this is expected behaviour.

The AWS collections are Python based. As far as I can tell, Ansible doesn't support running Python modules on Windows hosts (Ansible uses PowerShell for the various Windows modules):

[https://docs.ansible.com/ansible/latest/os_guide/windows_faq.html#can-i-run-python-modules-on-windows-hosts]

Can I run Python modules on Windows hosts?

No, the WinRM connection protocol is set to use PowerShell modules, so Python modules will not work. A way to bypass this issue to use delegate_to: localhost to run a Python module on the Ansible control node. This is useful if during a playbook, an external service needs to be contacted and there is no equivalent Windows module available.

The exception you've pasted only includes lines from the wrapper that Ansible uses to copy and then remotely execute modules, the s3_object code hasn't even been evaluated.

While some Ansible modules have PowerShell equivalents, this is mostly limited to the "core" modules. Given the level of effort that would be involved, I don't foresee PowerShell versions of the AWS modules being released any time soon.