Closed garethsaxby closed 2 years ago
ansibullbot
commented
3 years ago Files identified in the description:
lib/ansible/plugins/connection](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/lib/ansible/plugins/connection)If these files are inaccurate, please update the component name section of the description or use the !component bot command.
garethsaxby
commented
3 years ago Closing this temporarily as I think I need to revisit some of this first, apologies!
To be more specific; the errors are still happening, but my determination of the cause may not be right, and I want to dig into that more before someone else starts looking.
EDIT: I've reopened the issue now, as I've realised that I was just confusing myself a bit reading the fix I'd hacked together myself. The issue is still present when using the latest main from this repository.
ansibullbot
commented
3 years ago Files identified in the description:
plugins/connection/aws_ssm.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/connection/aws_ssm.py)If these files are inaccurate, please update the component name section of the description or use the !component bot command.
markuman
commented
3 years ago @garethsaxby thx for the detailed report!
Sadly I'm not familiar with ssm connection plugin.
If I force the plugin to use a regional endpoint for S3, and use a region when creating the client, as per my branch, it does work, albeit I'm not really sure -how- best to implement this to properly put a Pull Request together to fix the problem, given my branch feels like a really ugly hack.
There is a similar PR (not merged yet) that introduce a bucket_region parameter: https://github.com/ansible-collections/community.aws/pull/603/files#diff-8000d7fb9262e11565b61882567d729fe0958cd9b1a0105683d5bbe0e5b4e585R51
You can try to introduce that too
if bucket_region:
client = session.client(
service,
config=Config(signature_version="s3v4"),
endpoint_url=f'https://s3.{bucket_region}.amazonaws.com'
)
else:
client = session.client(
service,
config=Config(signature_version="s3v4")
)
atul-chegg
commented
3 years ago I also confirm this issue. I think a issue fix would be to use '-L' option with curl.
ryancabrera
commented
2 years ago I'm definitely having issues with bucket regions and encrypted buckets too.
bodnarbm
commented
2 years ago I hit this issue yesterday when attempting to use the aws ssm connection with using a newly created bucket in us-east-2 yesterday. Can we reopen this issue?
The referenced PR in the close action above (https://github.com/ansible-collections/community.aws/pull/1176) does not seem to resolve this particular issue. That PR was targeting a fix for pulling the region information for the S3 bucket used for SSM file transfers from the bucket metadata itself, but the pre-signed URLs generated for the S3 downloads are still pointing at the global S3 endpoint, and not the region specific one.
Thus, attempting to use the ssm plugin with a newly created transfer bucket in a region like us-east-2 continues to return the presigned URLs targeting the global S3 endpoint, which results in the 307 redirect to the regional endpoint, which then causes the presigned URL to fail with a signature mismatch error as the url was signed for the global endpoint and not the regional endpoint. I believe that #1190 needs to be further addressed to resolve this issue.
charles-paul-mox
commented
2 years ago Hi @bodnarbm please see https://github.com/ansible-collections/community.aws/pull/743/files to use virtual addressing. There is a PR from @phene but that relies on a hardcoded region to be defined.
bodnarbm
commented
2 years ago @charles-paul-mox Thank you, but that pr looks to be closed unmerged and I would prefer to not patch the plugin separately (if I was I would probably add the s3 client endpoint url as a separate variable, that way I could also get it to work with other endpoints also (like fips endpoints))
I'm hoping that someone like @tremble could reopen this issue though.
charles-paul-mox
commented
2 years ago Yes, I cannot merge PRs due to company policies. The virtual addressing is the important part.
phene
commented
2 years ago @charles-paul-mox My PR doesn't rely on a hard-coded region unless you are using a non-default partition like GovCloud. It uses the default global region just to query information about the S3 bucket's region, then uses the bucket's region from there on.
piotrkochan
commented
1 year ago Any real resolution to this problem? I'm using 5.1.0 release and there is still issue with AnsiballZ file:
sh-4.2$ cat AnsiballZ_yum.py
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>TemporaryRedirect</Code><Message>Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.</Message><Endpoint>testbucketkochan.s3.eu-west-3.amazonaws.com</Endpoint><Bucket>testbucketkochan</Bucket><RequestId>W5B7ZAZZ........</RequestId><HostId>DR//pSU97KgA8ZLVD/............................+lC/xwAuIBO/W4RuWIXqyFp+MZj0ZuI=</HostId><sh-4.2$
ThilinaPrasad
commented
1 year ago Any solution for this ?
ThilinaPrasad
commented
1 year ago Cause: This is mainly because of the TemporaryRedirect error from AWS s3 with the resigned URL. due to this when we transfer the setup.py file from host to remote it will download with below content on the remote.
<?xml version="1.0" encoding="UTF-8"?>
TemporaryRedirectto avoid this you can modify _get_boto_client() function client initializing as below to support addressing_style virtual for s3. Thanks! Feel free to correct me.
client = session.client( service, config=Config(signature_version="s3v4", s3={'addressing_style': 'virtual'}) )
Summary
When I try to execute a playbook against an Amazon Linux 2 instance in EC2 using the aws_ssm connection plugin and a recently created (less than an hour old) S3 bucket, it fails to correctly download
AnsiballZ_setup.py, resulting in a python syntax error" File \"/home/ssm-user/.ansible/tmp/ansible-tmp-1626190404.700778-20074-247496938615569/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r",.The curl is writing out the S3 XML error response to file, due to S3 returning a HTTP 307 redirect which the curl does not follow.
This HTTP 307 from S3 is expected, as per this AWS documentation, because the bucket is too new for the global S3 DNS to have propagated out yet, so a regional endpoint has to be used.
This overall seems similar to this issue, but is still happening for me when using the
mainbranch of this repository where the fix has been applied.I believe the underlying problem is that when the signed url is generated in the function
_file_transport_command, it is a global URL rather than a regional URL:For example, the URL below does not work and returns a 307; https://test-bucket-garethsaxby-20210713-153159.s3.amazonaws.com/i-089c1ec0c85524f5d//home/ssm-user/.ansible/tmp/ansible-tmp-1626189520.659307-19800-45563405192193/AnsiballZ_setup.py?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5QGXVMSCMPOQVZH3%2F20210713%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20210713T151841Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=18d520a539227540bef2ba06a6000dd6569c868aeb4cc6ae042fb895e5e2f880
Whilst the URL below, redirected by the 307, -does- work; https://test-bucket-garethsaxby-20210713-153159.s3.eu-west-2.amazonaws.com/i-089c1ec0c85524f5d//home/ssm-user/.ansible/tmp/ansible-tmp-1626189520.659307-19800-45563405192193/AnsiballZ_setup.py?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5QGXVMSCMPOQVZH3%2F20210713%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20210713T151841Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=18d520a539227540bef2ba06a6000dd6569c868aeb4cc6ae042fb895e5e2f880
If I force the plugin to use a regional endpoint for S3, and use a region when creating the client, as per my branch, it does work, albeit I'm not really sure -how- best to implement this to properly put a Pull Request together to fix the problem, given my branch feels like a really ugly hack.
Issue Type
Bug Report
Component Name
plugins/connection/aws_ssm
Ansible Version
Collection Versions
AWS SDK versions
Configuration
OS / Environment
Client: macOS Catalina 10.15.7, Ansible installed via Brew Remote: Amazon Linux 2, eu-west-2, ami-03ac5a9b225e99b02, amzn2-ami-hvm-2.0.20210701.0-x86_64-gp2
Steps to Reproduce
requirements.yml:
ansible.cfg:
inventory.aws_ec2.yml:
playbook.yml
Expected Results
I'm expecting the curl against the S3 signed URL on the remote host to pull down
AnsiballZ_setup.pycorrectly and continue running the playbook, returning the ping successfully.Actual Results
When I curl the signed URL from the remote instance using SSM Session Manager, I get the following response, showing that a 307 is being returned and I'm being redirected to the regional endpoint:
Code of Conduct