search

ansible-collections / amazon.aws

Ansible Collection for Amazon AWS
GNU General Public License v3.0
304 stars 334 forks source link

AWS S3 module Copy function does not seem to work #602

Closed AKSarav closed 2 years ago

AKSarav commented 2 years ago

Summary

I was referring to the Official documentation available on this page

aws_s3 documentation ansible

there was an example for copying objects from one bucket to another

While throws an exception as follows

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: UnboundLocalError: local variable 'tags' referenced before assignment fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 100, in <module>\n _ansiballz_main()\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.aws_s3', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.aws_s3', _modlib_path=modlib_path),\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 209, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1171, in <module>\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1165, in main\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 757, in copy_object_to_bucket\nUnboundLocalError: local variable 'tags' referenced before assignment\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

I have the latest version of Ansible. Ansible 11

My Boto Versions are above the recommended version on the article

boto3 1.17.58 botocore 1.20.58

Except for mode: copy all other modes seem to work properly.

Is this a bug or documentation is wrong. Can you clarify

here is the complete playbook

---
- name: Copy objects from one S3 bucket to another bucket using Ansible
  hosts: localhost

  tasks:
  - name: Copy Objects from one S3 bucket to another
    amazon.aws.aws_s3:
      bucket: destbucket
      profile: personal
      object: filename
      mode: copy
      copy_src: 
        bucket: sourcebucket
        object: filename
    register: copyresult

  - debug: msg="{{ copyresult.msg }}"

Issue Type

Bug Report

Component Name

amazon.aws.aws_s3

Ansible Version

$ ansible --version
ansible [core 2.11.6]
  config file = /Users/saravananthangaraj/.ansible.cfg
  configured module search path = ['/Users/saravananthangaraj/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/Cellar/ansible/4.8.0/libexec/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/saravananthangaraj/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.10.0 (default, Oct 13 2021, 06:44:31) [Clang 12.0.0 (clang-1200.0.32.29)]
  jinja version = 3.0.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.1
ansible.netcommon             2.4.0
ansible.posix                 1.3.0
ansible.utils                 2.4.2
ansible.windows               1.7.3
arista.eos                    2.2.0
awx.awx                       19.4.0
azure.azcollection            1.10.0
check_point.mgmt              2.1.1
chocolatey.chocolatey         1.1.0
cisco.aci                     2.1.0
cisco.asa                     2.1.0
cisco.intersight              1.0.17
cisco.ios                     2.5.0
cisco.iosxr                   2.5.0
cisco.meraki                  2.5.0
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.7.0
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.1.0
community.crypto              1.9.6
community.digitalocean        1.11.0
community.docker              1.10.0
community.fortios             1.0.0
community.general             3.8.1
community.google              1.0.0
community.grafana             1.2.3
community.hashi_vault         1.4.1
community.hrobot              1.2.0
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.1
community.mysql               2.3.1
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.5.0
community.proxysql            1.3.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.15.0
community.windows             1.7.0
community.zabbix              1.5.0
containers.podman             1.8.1
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.12.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.2
google.cloud                  1.0.2
hetzner.hcloud                1.6.0
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.3.0
junipernetworks.junos         2.6.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.9.0
netapp.cloudmanager           21.11.0
netapp.elementsw              21.6.1
netapp.ontap                  21.12.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.3.0
ngine_io.cloudstack           2.2.2
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.1
openvswitch.openvswitch       2.0.2
ovirt.ovirt                   1.6.4
purestorage.flasharray        1.11.0
purestorage.flashblade        1.7.0
sensu.sensu_go                1.12.0
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.23.0
theforeman.foreman            2.2.0
vyos.vyos                     2.6.0
wti.remote                    1.0.1

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /usr/local/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.17.58
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: ec2.py
---
Name: botocore
Version: 1.20.58
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
INVENTORY_ENABLED(/Users/saravananthangaraj/.ansible.cfg) = ['host_list', 'script', 'auto', 'yaml', 'ini', 'toml', 'aws_ec2']

OS / Environment

MAC OS CATALINA 10.15.7

Steps to Reproduce

---
- name: Copy objects from one S3 bucket to another bucket using Ansible
  hosts: localhost

  tasks:
  - name: Copy Objects from one S3 bucket to another
    amazon.aws.aws_s3:
      bucket: destbucket
      profile: personal
      object: filename
      mode: copy
      copy_src: 
        bucket: sourcebucket
        object: filename
    register: copyresult

  - debug: msg="{{ copyresult.msg }}"

Expected Results

The file should be from source bucket to destination bucket

Actual Results

Getting these exceptions

Exception 1

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756980.591985-99892-110059587658594/AnsiballZ_aws_s3.py\", line 100, in <module>\n _ansiballz_main()\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756980.591985-99892-110059587658594/AnsiballZ_aws_s3.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756980.591985-99892-110059587658594/AnsiballZ_aws_s3.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.aws_s3', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.aws_s3', _modlib_path=modlib_path),\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 209, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_u1ilxbza/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1171, in <module>\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_u1ilxbza/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1157, in main\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_u1ilxbza/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 418, in get_etag\n File \"/usr/local/Cellar/ansible/4.8.0/libexec/lib/python3.10/site-packages/botocore/client.py\", line 388, in _api_call\n return self._make_api_call(operation_name, kwargs)\n File \"/usr/local/Cellar/ansible/4.8.0/libexec/lib/python3.10/site-packages/botocore/client.py\", line 708, in _make_api_call\n raise error_class(parsed_response, operation_name)\nbotocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Exception 2

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: UnboundLocalError: local variable 'tags' referenced before assignment fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 100, in <module>\n _ansiballz_main()\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.aws_s3', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.aws_s3', _modlib_path=modlib_path),\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 209, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1171, in <module>\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1165, in main\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 757, in copy_object_to_bucket\nUnboundLocalError: local variable 'tags' referenced before assignment\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Code of Conduct

Akasurde commented 2 years ago

cc @abikouo Could you please take a look at this? Got introduce via https://github.com/ansible-collections/amazon.aws/pull/359

acelinkio commented 2 years ago

+1. Also encountering this error.

ansible 5.0.1 ansible-core 2.12.1 collection amazon.aws 3.0.0

abikouo commented 2 years ago

@AKSarav Thanks for taking the time to report this issue. The exception is raised when trying to copy a key not existing in the source bucket,

603 will fix this, could you please try it ?