search

ansible-collections / amazon.aws

Ansible Collection for Amazon AWS
GNU General Public License v3.0
309 stars 341 forks source link

aws_s3: SyntaxError: invalid syntax on RHEL 7 / CentOS 7 targets #1467

Closed asarubbo closed 1 year ago

asarubbo commented 1 year ago

Summary

Hi, by moving (in my case) from: ansible-6.7.0 / ansible-core-2.13.8 to ansible-7.4.0 / ansible-core-2.14.4

I'm unable to use anymore aws_s3 where the target machine is a CentOS 7

This is what I get by running ansible-playbook with -vvv:

Traceback (most recent call last):
  File "<stdin>", line 107, in <module>
  File "<stdin>", line 99, in _ansiballz_main
  File "<stdin>", line 48, in invoke_module
  File "/usr/lib64/python2.7/runpy.py", line 176, in run_module
    fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 411, in <module>
  File "/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 60, in <module>
  File "/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/botocore.py", line 339
    data_path = f"{os.environ['_ANSIBLE_PLACEBO_RECORD']}/{idx}"
                                                               ^
SyntaxError: invalid syntax
fatal: [my.host]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 107, in <module>\n  File \"<stdin>\", line 99, in _ansiballz_main\n  File \"<stdin>\", line 48, in invoke_module\n  File \"/usr/lib64/python2.7/runpy.py\", line 176, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/lib64/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File \"/usr/lib64/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py\", line 411, in <module>\n  File \"/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py\", line 60, in <module>\n  File \"/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/botocore.py\", line 339\n    data_path = f\"{os.environ['_ANSIBLE_PLACEBO_RECORD']}/{idx}\"\n                                                               ^\nSyntaxError: invalid syntax\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

I'm aware that README says "Due to the AWS SDK Python Support Policy this collection requires Python 3.6 or greater." but I decided to make a bug report considering that it affects RHEL 7 / CentOS 7 that does not provide by default the boto/botocore libraries compiled for python3

Issue Type

Bug Report

Component Name

aws_s3

Ansible Version

$ ansible --version
ansible [core 2.14.3]
  config file = /home/ago/.ansible.cfg
  configured module search path = ['/home/ago/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.10/site-packages/ansible
  ansible collection location = /home/ago/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/lib/python-exec/python3.10/ansible
  python version = 3.10.10 (main, Apr  3 2023, 12:46:35) [GCC 11.3.0] (/usr/bin/python3.10)
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
# /usr/lib/python3.10/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    5.2.0  
ansible.netcommon             4.1.0  
ansible.posix                 1.5.1  
ansible.utils                 2.9.0  
ansible.windows               1.13.0 
arista.eos                    6.0.0  
awx.awx                       21.12.0
azure.azcollection            1.14.0 
check_point.mgmt              4.0.0  
chocolatey.chocolatey         1.4.0  
cisco.aci                     2.4.0  
cisco.asa                     4.0.0  
cisco.dnac                    6.6.3  
cisco.intersight              1.0.23 
cisco.ios                     4.3.1  
cisco.iosxr                   4.1.0  
cisco.ise                     2.5.12 
cisco.meraki                  2.15.1 
cisco.mso                     2.2.1  
cisco.nso                     1.0.3  
cisco.nxos                    4.1.0  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.4  
community.aws                 5.2.0  
community.azure               2.0.0  
community.ciscosmb            1.0.5  
community.crypto              2.11.0 
community.digitalocean        1.23.0 
community.dns                 2.5.1  
community.docker              3.4.2  
community.fortios             1.0.0  
community.general             6.4.0  
community.google              1.0.0  
community.grafana             1.5.4  
community.hashi_vault         4.1.0  
community.hrobot              1.7.0  
community.libvirt             1.2.0  
community.mongodb             1.5.1  
community.mysql               3.6.0  
community.network             5.0.0  
community.okd                 2.3.0  
community.postgresql          2.3.2  
community.proxysql            1.5.1  
community.rabbitmq            1.2.3  
community.routeros            2.7.0  
community.sap                 1.0.0  
community.sap_libs            1.4.0  
community.skydive             1.0.0  
community.sops                1.6.1  
community.vmware              3.4.0  
community.windows             1.12.0 
community.zabbix              1.9.2  
containers.podman             1.10.1 
cyberark.conjur               1.2.0  
cyberark.pas                  1.0.17 
dellemc.enterprise_sonic      2.0.0  
dellemc.openmanage            6.3.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
dellemc.powerflex             1.5.0  
dellemc.unity                 1.5.0  
f5networks.f5_modules         1.22.1 
fortinet.fortimanager         2.1.7  
fortinet.fortios              2.2.2  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.1.2  
grafana.grafana               1.1.1  
hetzner.hcloud                1.10.0 
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       1.11.0 
infinidat.infinibox           1.3.12 
infoblox.nios_modules         1.4.1  
inspur.ispim                  1.3.0  
inspur.sm                     2.3.0  
junipernetworks.junos         4.1.0  
kubernetes.core               2.4.0  
lowlydba.sqlserver            1.3.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0 
netapp.ontap                  22.3.0 
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.4.0  
netbox.netbox                 3.11.0 
ngine_io.cloudstack           2.3.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.3  
openstack.cloud               1.10.0 
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.4.1  
purestorage.flasharray        1.17.0 
purestorage.flashblade        1.10.0 
purestorage.fusion            1.3.0  
sensu.sensu_go                1.13.2 
splunk.es                     2.1.0  
t_systems_mms.icinga_director 1.32.0 
theforeman.foreman            3.9.0  
vmware.vmware_rest            2.2.0  
vultr.cloud                   1.7.0  
vyos.vyos                     4.0.0  
wti.remote                    1.0.4

AWS SDK versions

$ pip show boto boto3 botocore

I don't have pip, but I can assure I have:
boto3-1.26.114
botocore-1.29.114

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = /home/ago/.ansible.cfg
DEFAULT_FORKS(/home/ago/.ansible.cfg) = 30
DEFAULT_HOST_LIST(/home/ago/.ansible.cfg) = ['/home/ago/git/ansible/common/hosts']
DEFAULT_REMOTE_USER(/home/ago/.ansible.cfg) = root
DEPRECATION_WARNINGS(/home/ago/.ansible.cfg) = False
HOST_KEY_CHECKING(/home/ago/.ansible.cfg) = False
INTERPRETER_PYTHON(/home/ago/.ansible.cfg) = auto_legacy_silent

OS / Environment

Target machine is a up-to-date CentOS 7: CentOS Linux release 7.9.2009 (Core)

The host where ansible is running is an up-to-date Gentoo.

Steps to Reproduce

      aws_s3:
        aws_access_key: "{{ aws_access_key }}"
        aws_secret_key: "{{ aws_secret_key }}"
        s3_url: "{{ s3_url }}"
        bucket: "{{ website_bucket }}"
        object: "/file"
        dest: "/tmp/file"
        mode: get
        overwrite: different
        aws_region: "{{ aws_region }}"

Expected Results

I expect to download the file

Actual Results

Traceback (most recent call last):
  File "<stdin>", line 107, in <module>
  File "<stdin>", line 99, in _ansiballz_main
  File "<stdin>", line 48, in invoke_module
  File "/usr/lib64/python2.7/runpy.py", line 176, in run_module
    fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 411, in <module>
  File "/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 60, in <module>
  File "/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/botocore.py", line 339
    data_path = f"{os.environ['_ANSIBLE_PLACEBO_RECORD']}/{idx}"
                                                               ^
SyntaxError: invalid syntax
fatal: [my.host]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 107, in <module>\n  File \"<stdin>\", line 99, in _ansiballz_main\n  File \"<stdin>\", line 48, in invoke_module\n  File \"/usr/lib64/python2.7/runpy.py\", line 176, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/lib64/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File \"/usr/lib64/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py\", line 411, in <module>\n  File \"/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py\", line 60, in <module>\n  File \"/tmp/ansible_aws_s3_payload_VCFjMg/ansible_aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/botocore.py\", line 339\n    data_path = f\"{os.environ['_ANSIBLE_PLACEBO_RECORD']}/{idx}\"\n                                                               ^\nSyntaxError: invalid syntax\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Code of Conduct

tremble commented 1 year ago

@asarubbo,

Thanks for taking the time to open this Issue. The AWS SDK that this collection depends on has dropped support for Python <3.6. As a result, with release 2.0.0 we dropped support for Python < 3.6. The specific code you're having problems with was introduced in release 5.0.0, so you may be able to use the 4.x releases of this collection for your CentOS 7 boxes. However, if you wish to use newer versions you'll need to use Python 3.6 and install a supported version of botocore.

While CentOS has built python-botocore-1.12.75-1.el7, this is also a version of botocore which the collection no longer supports.

The only exception where we're maintaining Python 2.7 support is the ec2_metadata_facts module.