search

ansible-collections / amazon.aws

Ansible Collection for Amazon AWS
GNU General Public License v3.0
304 stars 334 forks source link

aws_s3 Folder Creation issue on S3 Buckets with ACLs disabled #1137

Closed jrglynn2 closed 1 year ago

jrglynn2 commented 1 year ago

Summary

When I try to create a folder using aws_s3 on an S3 bucket with ACLs disabled, I get an error UnboundLocalError: local variable 'object_acl' referenced before assignment.

Issue Type

Bug Report

Component Name

aws_s3

Ansible Version

$ ansible --version
Ansible Full Version:  6.4.0
Ansible-Core Version:  ansible [core 2.13.4]
  config file = /opt/ansible/ansible.cfg
  configured module search path = ['/home/ec2-user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ec2-user/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /opt/ansible/ansible_collections
  executable location = /home/ec2-user/.local/bin/ansible
  python version = 3.9.7 (default, Sep 13 2021, 08:18:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
  jinja version = 3.0.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection                    Version
----------------------------- -------
amazon.aws                    3.4.0
ansible.netcommon             3.1.1
ansible.posix                 1.4.0
ansible.utils                 2.6.1
ansible.windows               1.11.1
arista.eos                    5.0.1
awx.awx                       21.5.0
azure.azcollection            1.13.0
check_point.mgmt              2.3.0
chocolatey.chocolatey         1.3.0
cisco.aci                     2.2.0
cisco.asa                     3.1.0
cisco.dnac                    6.6.0
cisco.intersight              1.0.19
cisco.ios                     3.3.1
cisco.iosxr                   3.3.1
cisco.ise                     2.5.3
cisco.meraki                  2.11.0
cisco.mso                     2.0.0
cisco.nso                     1.0.3
cisco.nxos                    3.1.1
cisco.ucs                     1.8.0
cloud.common                  2.1.2
cloudscale_ch.cloud           2.2.2
community.aws                 3.5.0
community.azure               1.1.0
community.ciscosmb            1.0.5
community.crypto              2.5.0
community.digitalocean        1.21.0
community.dns                 2.3.2
community.docker              2.7.1
community.fortios             1.0.0
community.general             5.6.0
community.google              1.0.0
community.grafana             1.5.2
community.hashi_vault         3.2.0
community.hrobot              1.5.2
community.kubernetes          *
community.libvirt             1.2.0
community.mongodb             1.4.2
community.mysql               3.5.1
community.network             4.0.1
community.okd                 2.2.0
community.postgresql          2.2.0
community.proxysql            1.4.0
community.rabbitmq            1.2.2
community.routeros            2.3.0
community.sap                 1.0.0
community.sap_libs            1.3.0
community.skydive             1.0.0
community.sops                1.4.0
community.vmware              2.9.1
community.windows             1.11.0
community.zabbix              1.8.0
containers.podman             1.9.4
cyberark.conjur               1.2.0
cyberark.pas                  1.0.14
dellemc.enterprise_sonic      1.1.2
dellemc.openmanage            5.5.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.19.0
fortinet.fortimanager         2.1.5
fortinet.fortios              2.1.7
frr.frr                       2.0.0
gluster.gluster               1.0.2
google.cloud                  1.0.2
hetzner.hcloud                1.8.2
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.9.0
infinidat.infinibox           1.3.3
infoblox.nios_modules         1.3.0
inspur.ispim                  1.0.1
inspur.sm                     2.0.0
junipernetworks.junos         3.1.0
kubernetes.core               2.3.2
mellanox.onyx                 1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.19.0
netapp.elementsw              21.7.0
netapp.ontap                  21.23.0
netapp.storagegrid            21.11.0
netapp.um_info                21.8.0
netapp_eseries.santricity     1.3.1
netbox.netbox                 3.7.1
ngine_io.cloudstack           2.2.4
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.2
openstack.cloud               1.9.1
openvswitch.openvswitch       2.1.0
ovirt.ovirt                   2.2.3
purestorage.flasharray        1.13.0
purestorage.flashblade        1.10.0
purestorage.fusion            1.1.0
sensu.sensu_go                1.13.1
servicenow.servicenow         1.0.6
splunk.es                     2.1.0
t_systems_mms.icinga_director 1.31.0
theforeman.foreman            3.6.0
vmware.vmware_rest            2.2.0
vultr.cloud                   1.1.0
vyos.vyos                     3.0.1
wti.remote                    1.0.4

# /opt/ansible/ansible_collections
Collection       Version
---------------- -------
amazon.aws       5.0.1
ansible.posix    1.4.0
community.aws    5.0.0
community.crypto 2.7.0

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.24.80
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/ec2-user/.local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.27.80
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/ec2-user/.local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
COLLECTIONS_PATHS(/opt/ansible/ansible.cfg) = ['/opt/ansible/ansible_collections']
DEFAULT_FORKS(/opt/ansible/ansible.cfg) = 50
DEFAULT_HOST_LIST(/opt/ansible/ansible.cfg) = ['/opt/ansible/inventory/dynamic']
DEFAULT_LOG_PATH(/opt/ansible/ansible.cfg) = /var/log/ansible.log
DEFAULT_PRIVATE_KEY_FILE(/opt/ansible/ansible.cfg) = /opt/ansible/keys/ansible.pem
DEFAULT_REMOTE_USER(/opt/ansible/ansible.cfg) = ansible
DEFAULT_ROLES_PATH(/opt/ansible/ansible.cfg) = ['/opt/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/opt/ansible/ansible.cfg) = yaml
HOST_KEY_CHECKING(/opt/ansible/ansible.cfg) = False

OS / Environment

Red Hat Enterprise Linux release 8.6 (Ootpa)

Steps to Reproduce

-- Play:
# Create S3 Folders
- name: create s3 folders for 1st SFTP ID - /jrglynn2
  delegate_to: localhost
  environment:
    AWS_PROFILE: "ansible-devops"
  amazon.aws.aws_s3:
    bucket: bucket_acls_disabled
    object: "{{ item }}"
    mode: create
    permission: bucket-owner-full-control
    purge_tags: false
  loop:
    - "/jrglynn2"

Expected Results

I expect the folder jrglynn2 to be created on S3 Bucket bucket_acls_disabled

Actual Results

-- Verbose Error (run with -vvv)
task path: /opt/ansible/roles/sftp_transfer_config/tasks/main.yml:6
redirecting (type: modules) amazon.aws.aws_s3 to amazon.aws.s3_object
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: ec2-user
<localhost> EXEC /bin/sh -c 'echo ~ec2-user && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ec2-user/.ansible/tmp `"&& mkdir "` echo /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921 `" && echo ansible-tmp-1665089009.6957664-129681-268809806524921="` echo /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921 `" ) && sleep 0'
redirecting (type: modules) amazon.aws.aws_s3 to amazon.aws.s3_object
Using module file /opt/ansible/ansible_collections/amazon/aws/plugins/modules/s3_object.py
<localhost> PUT /home/ec2-user/.ansible/tmp/ansible-local-129638dpd3owbt/tmpcwwanxip TO /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/ /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py && sleep 0'
<localhost> EXEC /bin/sh -c 'AWS_PROFILE=ansible-devops /usr/bin/python3 /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/ > /dev/null 2>&1 && sleep 0'
redirecting (type: modules) amazon.aws.aws_s3 to amazon.aws.s3_object
The full traceback is:
Traceback (most recent call last):
  File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 107, in <module>
    _ansiballz_main()
  File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
  File "/usr/lib64/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1276, in <module>
  File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1221, in main
UnboundLocalError: local variable 'object_acl' referenced before assignment
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
redirecting (type: action) amazon.aws.aws_s3 to amazon.aws.s3_object
<localhost> EXEC /bin/sh -c 'echo ~ec2-user && sleep 0'
failed: [asd01-srv-jrglynn2 -> localhost] (item=/jrglynn2) => changed=false
  ansible_loop_var: item
  item: /jrglynn2
  module_stderr: |-
    Traceback (most recent call last):
      File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 107, in <module>
        _ansiballz_main()
      File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 99, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/ec2-user/.ansible/tmp/ansible-tmp-1665089009.6957664-129681-268809806524921/AnsiballZ_s3_object.py", line 47, in invoke_module
        runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.s3_object', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.s3_object', _modlib_path=modlib_path),
      File "/usr/lib64/python3.9/runpy.py", line 210, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1276, in <module>
      File "/tmp/ansible_amazon.aws.aws_s3_payload_ag1tgsni/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1221, in main
    UnboundLocalError: local variable 'object_acl' referenced before assignment
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

Code of Conduct

ansibullbot commented 1 year ago

Files identified in the description: None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

Tyler-2 commented 1 year ago

Is this problem back? I don't see the except is_boto3_error_code('AccessControlListNotSupported'): in the module anymore.

tremble commented 1 year ago

This specific issue shouldn't be back (undefined variable with ACLs disabled). The code was actually re-written, that variable doesn't exist any more, and the replacement is more careful to check ACLs are enabled before trying to access/set them.