search

ansible-collections / amazon.aws

Ansible Collection for Amazon AWS
GNU General Public License v3.0
309 stars 341 forks source link

S3 Presigned URL format #403

Closed castironclay closed 3 years ago

castironclay commented 3 years ago

Summary

I am seeing a difference in the URL format when generating presigned URLs with awscli versus Ansible amazon.aws.aws_s3 module. The URL generated with Ansible is failing to authenticate. Below is the error I am seeing. It appears that newer presigned urls contain additional arguments that are missing from a url generated by Ansible. 

The request signature we calculated does not match the signature you provided. Check your key and signing method.

Issue Type

Bug Report

Component Name

amazon.aws.aws_s3

Ansible Version

$ ansible --version
ansible [core 2.11.2] 
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/dist-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
  jinja version = 3.0.1
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection Version
---------- -------
amazon.aws 1.5.0  

# /usr/local/lib/python3.7/dist-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0  
ansible.netcommon             2.2.0  
ansible.posix                 1.2.0  
ansible.utils                 2.3.0  
ansible.windows               1.7.0  
arista.eos                    2.2.0  
awx.awx                       19.2.2 
azure.azcollection            1.7.0  
check_point.mgmt              2.0.0  
chocolatey.chocolatey         1.1.0  
cisco.aci                     2.0.0  
cisco.asa                     2.0.2  
cisco.intersight              1.0.15 
cisco.ios                     2.3.0  
cisco.iosxr                   2.3.0  
cisco.meraki                  2.4.2  
cisco.mso                     1.2.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.4.0  
cisco.ucs                     1.6.0  
cloudscale_ch.cloud           2.2.0  
community.aws                 1.5.0  
community.azure               1.0.0  
community.crypto              1.7.1  
community.digitalocean        1.7.0  
community.docker              1.8.0  
community.fortios             1.0.0  
community.general             3.3.0  
community.google              1.0.0  
community.grafana             1.2.1  
community.hashi_vault         1.3.0  
community.hrobot              1.1.1  
community.kubernetes          1.2.1  
community.kubevirt            1.0.0  
community.libvirt             1.0.1  
community.mongodb             1.2.1  
community.mysql               2.1.0  
community.network             3.0.0  
community.okd                 1.1.2  
community.postgresql          1.3.0  
community.proxysql            1.0.0  
community.rabbitmq            1.0.3  
community.routeros            1.2.0  
community.skydive             1.0.0  
community.sops                1.1.0  
community.vmware              1.11.0 
community.windows             1.5.0  
community.zabbix              1.3.0  
containers.podman             1.6.1  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.7  
dellemc.enterprise_sonic      1.1.0  
dellemc.openmanage            3.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.10.1 
fortinet.fortimanager         2.1.2  
fortinet.fortios              2.1.1  
frr.frr                       1.0.3  
gluster.gluster               1.0.1  
google.cloud                  1.0.2  
hetzner.hcloud                1.4.3  
hpe.nimble                    1.1.3  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.2.4  
inspur.sm                     1.2.0  
junipernetworks.junos         2.3.0  
kubernetes.core               1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.2.0 
netapp.azure                  21.7.0 
netapp.cloudmanager           21.7.0 
netapp.elementsw              21.6.1 
netapp.ontap                  21.7.0 
netapp.um_info                21.6.0 
netapp_eseries.santricity     1.2.13 
netbox.netbox                 3.1.1  
ngine_io.cloudstack           2.1.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.0  
openstack.cloud               1.5.0  
openvswitch.openvswitch       2.0.0  
ovirt.ovirt                   1.5.3  
purestorage.flasharray        1.8.0  
purestorage.flashblade        1.6.0  
sensu.sensu_go                1.11.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.18.0 
theforeman.foreman            2.1.1  
vyos.vyos                     2.3.1  
wti.remote                    1.0.1

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /usr/local/lib/python3.7/dist-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.17.110
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.7/dist-packages
Requires: s3transfer, jmespath, botocore
Required-by: 
---
Name: botocore
Version: 1.20.110
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.7/dist-packages
Requires: python-dateutil, jmespath, urllib3
Required-by: s3transfer, boto3

Configuration

- hosts: localhost
  gather_facts: no
  tasks:
    - name: Put object
      amazon.aws.aws_s3:
        bucket: "82be73b"
        object: wg3-client.conf
        src: /root/wg3-client.conf
        mode: put
        headers: "X-Amz-Algorithm=AWS4-HMAC-SHA256"
      register: bucket_put
    - debug:
        msg: "{{ bucket_put.url }}"

OS / Environment

Debian

Steps to Reproduce

---
- hosts: localhost
  gather_facts: no
  tasks:
    - name: Put object
      amazon.aws.aws_s3:
        bucket: "82be73b"
        object: wg3-client.conf
        src: /root/wg3-client.conf
        mode: put
      register: bucket_put
    - debug:
        msg: "{{ bucket_put.url }}"

Expected Results

I expect an ansible debug message showing a valid link.

Valid link generated using awscli

https://82be73b.s3.us-east-1.amazonaws.com/wg3-client.conf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=CHANGEME%2F20210712%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210712T194031Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=19ca0d97e3b808a89a480c3bdc86a145a0cc68c8e044841b31e385c6e1a1ff6f

Invalid link generated from url return value of the aws_s3 module

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "https://82be73b.s3.amazonaws.com/wg3-client.conf?AWSAccessKeyId=CHANGEME&Signature=Qjy271BkaCEW4GSTQmsXjQ8NiV0%3D&Expires=1626120114"
}

Actual Results

Error below is seen when trying to follow link generated by Ansible.

The request signature we calculated does not match the signature you provided. Check your key and signing method.

Code of Conduct

ansibullbot commented 3 years ago

Files identified in the description: None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

castironclay commented 3 years ago

I've just realize my confusion. The URL that is generated is used to perform the POST of the object. I can close this issue.