Closed mjharvey closed 1 year ago
Let me know how you invoke awscurl, and also how do you specify the credentials (environment variable, command line, etc).
And thanks for reporting the issue.
On Tue, May 2, 2023 at 3:35 PM Matt Harvey @.***> wrote:
Hi. I have been using awscurl on an EC2 instance to call an API endpoint that is secured via IAM permissions. This worked fine under 0.26.0, but with the release of 0.27.0 I am now encountering this error
File "/usr/local/bin/awscurl", line 11, in
sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 543, in main inner_main(sys.argv[1:]) File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 508, in inner_main args.profile) File "/usr/local/lib/python3.6/site-packages/awscurl/awscurl.py", line 427, in load_aws_config cred = session.get_credentials() File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 449, in get_credentials 'credential_provider').load_credentials() File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 897, in get_component self._components[name] = factory() File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 176, in lambda: botocore.credentials.create_credential_resolver(self)) File "/usr/local/lib/python3.6/site-packages/botocore/credentials.py", line 55, in create_credential_resolver metadata_timeout = session.get_config_variable('metadata_service_timeout') File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 265, in get_config_variable elif self._found_in_config_file(methods, var_config): File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 286, in _found_in_config_file return var_config[0] in self.get_scoped_config() File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 358, in get_scoped_config raise ProfileNotFound(profile=profile_name) botocore.exceptions.ProfileNotFound: The config profile (default) could not be found The problem looks like it might have been caused by this change: 7b38c7f ?diff=split?diff=split?diff=split%3Fdiff%3Dsplit https://github.com/okigan/awscurl/commit/7b38c7f438a8f1714be188f1b024a5d5fc1ea26d?diff=split?diff=split?diff=split%3Fdiff%3Dsplit
Is this a known issue, or this there a workaround for the problem I could apply?
Thanks.
— Reply to this email directly, view it on GitHub https://github.com/okigan/awscurl/issues/163, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADUYXVYESPMQBZQX2BWT3TXEFV2RANCNFSM6AAAAAAXTRIPQM . You are receiving this because you are subscribed to this thread.Message ID: @.***>
There are no environment variables or profile setup on the EC2 instance for authentication. What I am guessing was happening previously was the session = botocore.session.get_session()
gets credentials for the execution role associated with the EC2 instance (I may be wrong though).
Alright I’ll recheck on ec2 instance. Which service are you accessing (ec2, lambda..)?On May 2, 2023, at 6:22 PM, Matt Harvey @.***> wrote: There are no environment variables or profile setup on the EC2 instance for authentication. What I am guessing was happening previously was the session = botocore.session.get_session() gets credentials for the execution role associated with the EC2 instance (I may be wrong though).
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>
Lambda via API Gateway endpoint. The API gateway endpoint is secured via IAM policy that restricts access based on AWS Organisation Units.
From: Igor Okulist @.> Sent: Wednesday, 3 May 2023 11:42 am To: okigan/awscurl @.> Cc: Matt Harvey @.>; Author @.> Subject: Re: [okigan/awscurl] Credentials issue introduced in 0.27.0 when running on EC2 instance. (Issue #163)
Alright I’ll recheck on ec2 instance. Which service are you accessing (ec2, lambda..)?On May 2, 2023, at 6:22 PM, Matt Harvey @.***> wrote: There are no environment variables or profile setup on the EC2 instance for authentication. What I am guessing was happening previously was the session = botocore.session.get_session() gets credentials for the execution role associated with the EC2 instance (I may be wrong though).
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>
— Reply to this email directly, view it on GitHubhttps://github.com/okigan/awscurl/issues/163#issuecomment-1532284302, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABHSEFSDHKDNVKBKOIMUKVDXEGLUTANCNFSM6AAAAAAXTRIPQM. You are receiving this because you authored the thread.Message ID: @.***>
got pulled in with other items -- will review asap.
Capturing nice way to repro in aws cloud shell:
[cloudshell-user@ip-10-4-127-13 ~]$ pip3 install -q awscurl==0.26
[cloudshell-user@ip-10-4-127-13 ~]$ awscurl --service s3 https://awscurl-sample-bucket.s3.amazonaws.com
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>awscurl-sample-bucket</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>awscurl-sample-file.txt</Key><LastModified>2017-07-25T21:27:38.000Z</LastModified><ETag>"d41d8cd98f00b204e9800998ecf8427e"</ETag><Size>0</Size><Owner><ID>978427f78b00827efacb8fe2bd55ea30cbcb1d228cd3758972314cb67e763402</ID><DisplayName>okigan</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents></ListBucketResult>
[cloudshell-user@ip-10-4-127-13 ~]$ pip3 install -q awscurl==0.27
[cloudshell-user@ip-10-4-127-13 ~]$ awscurl --service s3 https://awscurl-sample-bucket.s3.amazonaws.com
Traceback (most recent call last):
File "/home/cloudshell-user/.local/bin/awscurl", line 8, in <module>
sys.exit(main())
File "/home/cloudshell-user/.local/lib/python3.7/site-packages/awscurl/awscurl.py", line 543, in main
inner_main(sys.argv[1:])
File "/home/cloudshell-user/.local/lib/python3.7/site-packages/awscurl/awscurl.py", line 508, in inner_main
args.profile)
File "/home/cloudshell-user/.local/lib/python3.7/site-packages/awscurl/awscurl.py", line 427, in load_aws_config
cred = session.get_credentials()
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 508, in get_credentials
'credential_provider'
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 1108, in get_component
self._components[name] = factory()
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 187, in _create_credential_resolver
self, region_name=self._last_client_region_used
File "/usr/local/lib/python3.7/site-packages/botocore/credentials.py", line 74, in create_credential_resolver
metadata_timeout = session.get_config_variable('metadata_service_timeout')
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 318, in get_config_variable
logical_name
File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 426, in get_config_variable
return provider.provide()
File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 628, in provide
value = provider.provide()
File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 718, in provide
scoped_config = self._session.get_scoped_config()
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 416, in get_scoped_config
raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (default) could not be found
[cloudshell-user@ip-10-4-127-13 ~]$
Output from the latest release:
[cloudshell-user@ip-10-2-12-122 ~]$ pip3 install -q awscurl==0.28
[cloudshell-user@ip-10-2-12-122 ~]$ awscurl --service s3 https://awscurl-sample-bucket.s3.amazonaws.com
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>awscurl-sample-bucket</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Contents><Key>awscurl-sample-file.txt</Key><LastModified>2017-07-25T21:27:38.000Z</LastModified><ETag>"d41d8cd98f00b204e9800998ecf8427e"</ETag><Size>0</Size><Owner><ID>978427f78b00827efacb8fe2bd55ea30cbcb1d228cd3758972314cb67e763402</ID><DisplayName>okigan</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents><Contents><Key>awscurl-sample-file:.txt</Key><LastModified>2023-05-05T16:03:34.000Z</LastModified><ETag>"d41d8cd98f00b204e9800998ecf8427e"</ETag><Size>0</Size><Owner><ID>978427f78b00827efacb8fe2bd55ea30cbcb1d228cd3758972314cb67e763402</ID><DisplayName>okigan</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents></ListBucketResult>
Hi. I have been using awscurl on an EC2 instance to call an API endpoint that is secured via IAM permissions. This worked fine under 0.26.0, but with the release of 0.27.0 I am now encountering this error
The problem looks like it might have been caused by this change: https://github.com/okigan/awscurl/commit/7b38c7f438a8f1714be188f1b024a5d5fc1ea26d?diff=split?diff=split?diff=split%3Fdiff%3Dsplit
Is this a known issue, or this there a workaround for the problem I could apply?
Thanks.