search

timkay / aws

Easy command line access to Amazon EC2, S3, SQS, ELB, and SDB
http://timkay.com/aws/
423 stars 122 forks source link

Support for eu-central-1 and IAM role authentication #97

Open mattheworiordan opened 9 years ago

mattheworiordan commented 9 years ago

I believe there has been a protocol change for IAM auth that is required in eu-central-1 that is incompatible with the current version of aws. See below:

$ aws describe-tags --region eu-central-1
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+

This command works fine for other regions, and works with the official AWS cli tool.

mattheworiordan commented 9 years ago

FYI, see https://forums.aws.amazon.com/thread.jspa?messageID=582707

timkay commented 9 years ago

I recall implementing AWS V4 signatures. Please try adding --AWS4 to the command line. I don't recall if the support is for all AWS products.

mattheworiordan commented 9 years ago

Sorry, no, that doesn't work:

$ ./aws describe-tags --region eu-central-1 --AWS4
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+
timkay commented 9 years ago

I just updated the code with support for Signature Version 4 across almost all other services. (S3 was already supported. Now EC2, etc.) Feedback, please.

To use, add --AWS4 to all commands or in your ~/.awsrc file.

Should it become the default?

mattheworiordan commented 9 years ago

Hi @timkay

I am not sure if I am doing something wrong, but I downloaded the latest version and tried it with --AWS4 and I see the same error unfortunately.

$ wget https://raw.github.com/timkay/aws/master/aws

$ ./aws describe-tags --region eu-central-1 --AWS4
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+

Am I doing something wrong? When I remove --region eu-central-1 it works

timkay commented 9 years ago

Hmmm. Works for me:

./aws describe-tags --region eu-central-1 --AWS4

<?xml version="1.0" encoding="UTF-8"?>

f9b4a3b4-cb79-4cfa-be74-cfff9d6379fb

Please send the output with -vv (two v's).

mattheworiordan commented 9 years ago

Here is the output

$ ./aws describe-tags --region eu-central-1 --AWS4 -vv
aws versions: (ec2: 2013-10-15, sqs: 2012-11-05, elb: 2011-11-15, sdb: 2009-04-15, iam: 2010-05-08, ebn: 2010-12-01, cfn: 2010-05-15, rds: 2013-09-09)
curl version: 7.35.0
HTTP/1.1 200 OK
x-amz-id-2: 0k7SCYmB9hOIhOl2JVzJla+ZTRc0qp4pce0Hy+iiJ6V3IsTchPgFqdZaAYaZJZAL
x-amz-request-id: 29CF96BB304ECD8C
Date: Tue, 14 Jul 2015 09:11:31 GMT
Last-Modified: Tue, 17 Mar 2009 15:15:37 GMT
ETag: "4108ecce80045c0c38bbc77a3bc600e5"
Accept-Ranges: bytes
Content-Type: text/plain
Content-Length: 28
Server: AmazonS3

aws sanity-check succeeded!
ec2(Action, DescribeTags)
data = []
https://ec2.eu-central-1.amazonaws.com/?AWSAccessKeyId=ASIAJ334H7HOLAZGIHQA&Action=DescribeTags&Expires=2015-07-14T09%253A12%253A01Z&SecurityToken=AQoDYXdzENL%252F%252F%252F%252F%252F%252F%252F%252F%252F%252FwEa
4AMni%252BntH7r0r9TLrQg5hKTS4c2eUf0Pf%252BaPQg3u5o9KHd4C3iueXssGrpRiG%252F%252BF%252FvQR37cMa2Kkd929d6wFgVgB6v9Uno4nJk2%252BNkkoUCbyuzcmu7jO2lHZg%252BDDXK1znsKHh9QGEnJpbpI%252Fm3cZEXgHpJj%252BmI9D3gGJVbuVYvtpThO
30evfDRvEK6eMY9oAa4fvgxnPTZfAz5t9mBO8aepd0ER8dteu8ChC6tWqfU3hf7XTm0zdofMvsj2yqyxWx9P%252BxbcH%252Fdz47ddCPtPcUKadereOHGjZQVgyKkLLLFZ855DvXwemW%252BYAVdCKZO9F%252F%252BKVXBQASDBbr8rkU38s3ozuSNZDk4%252FEoX7SO8%252
FcMf4lWbWArC%252BL%252BH1aWnFxahpEiV16OMCshWQyQe1IF9JGqx5OEJCPnA7nievfNlN9IM%252FofLBX8Sz4LV3U0zV5hvIRxVeUEWnBBPQGL65Iqwpj0KcxrNGNb2W1lmGDLBZhXEkm1f0zXCFuq9amkT8uaOwj28iGoky7uAepM8jXyRr7%252FW8XsiYi0PhRULIJI%252
BUKDeQZAbuNHjZnARzobYNhLmyi07E%252BbFOR5VGp%252FWHgjCpJw5eBpgGhGDaUaGzHIxI8s0rBMkDLhnjv1G1qm3keKPe%252FJH0gkqKTrQU%253D&SignatureMethod=HmacSHA1&SignatureVersion=4&Version=2013-10-15&X-Amz-Algorithm=AWS4-HMAC-SH
A256&X-Amz-Credential=ASIAJ334H7HOLAZGIHQA%2F20150714%2Feu-central-1%2Fec2%2Faws4_request&X-Amz-Date=20150714T091130Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=d6440df17a61954a4f670190d5909e7d4bb
faab5073dcd9434b058b1866b396f
cmd=[curl  -q -g -S --remote-time --retry 3 --verbose -s  'https://ec2.eu-central-1.amazonaws.com/?AWSAccessKeyId=ASIAJ334H7HOLAZGIHQA&Action=DescribeTags&Expires=2015-07-14T09%253A12%253A01Z&SecurityToken=AQoDY
XdzENL%252F%252F%252F%252F%252F%252F%252F%252F%252F%252FwEa4AMni%252BntH7r0r9TLrQg5hKTS4c2eUf0Pf%252BaPQg3u5o9KHd4C3iueXssGrpRiG%252F%252BF%252FvQR37cMa2Kkd929d6wFgVgB6v9Uno4nJk2%252BNkkoUCbyuzcmu7jO2lHZg%252BDD
XK1znsKHh9QGEnJpbpI%252Fm3cZEXgHpJj%252BmI9D3gGJVbuVYvtpThO30evfDRvEK6eMY9oAa4fvgxnPTZfAz5t9mBO8aepd0ER8dteu8ChC6tWqfU3hf7XTm0zdofMvsj2yqyxWx9P%252BxbcH%252Fdz47ddCPtPcUKadereOHGjZQVgyKkLLLFZ855DvXwemW%252BYAVdC
KZO9F%252F%252BKVXBQASDBbr8rkU38s3ozuSNZDk4%252FEoX7SO8%252FcMf4lWbWArC%252BL%252BH1aWnFxahpEiV16OMCshWQyQe1IF9JGqx5OEJCPnA7nievfNlN9IM%252FofLBX8Sz4LV3U0zV5hvIRxVeUEWnBBPQGL65Iqwpj0KcxrNGNb2W1lmGDLBZhXEkm1f0zXC
Fuq9amkT8uaOwj28iGoky7uAepM8jXyRr7%252FW8XsiYi0PhRULIJI%252BUKDeQZAbuNHjZnARzobYNhLmyi07E%252BbFOR5VGp%252FWHgjCpJw5eBpgGhGDaUaGzHIxI8s0rBMkDLhnjv1G1qm3keKPe%252FJH0gkqKTrQU%253D&SignatureMethod=HmacSHA1&Signatu
reVersion=4&Version=2013-10-15&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ334H7HOLAZGIHQA%2F20150714%2Feu-central-1%2Fec2%2Faws4_request&X-Amz-Date=20150714T091130Z&X-Amz-Expires=30&X-Amz-SignedHeade
rs=host&X-Amz-Signature=d6440df17a61954a4f670190d5909e7d4bbfaab5073dcd9434b058b1866b396f']
* Hostname was NOT found in DNS cache
*   Trying 54.239.54.28...
* Connected to ec2.eu-central-1.amazonaws.com (54.239.54.28) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using AES128-SHA
* Server certificate:
*        subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=ec2.eu-central-1.amazonaws.com
*        start date: 2015-06-18 00:00:00 GMT
*        expire date: 2015-10-18 23:59:59 GMT
*        subjectAltName: ec2.eu-central-1.amazonaws.com matched
*        issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of use at https://www.verisign.com/rpa (c)10; CN=VeriSign Class 3 Secure Server CA - G3
*        SSL certificate verify ok.
> GET /?AWSAccessKeyId=ASIAJ334H7HOLAZGIHQA&Action=DescribeTags&Expires=2015-07-14T09%253A12%253A01Z&SecurityToken=AQoDYXdzENL%252F%252F%252F%252F%252F%252F%252F%252F%252F%252FwEa4AMni%252BntH7r0r9TLrQg5hKTS4c2eUf0Pf%252BaPQg3u5o9KHd4C3iueXssGrpRiG%252F%252BF%252FvQR37cMa2Kkd929d6wFgVgB6v9Uno4nJk2%252BNkkoUCbyuzcmu7jO2lHZg%252BDDXK1znsKHh9QGEnJpbpI%252Fm3cZEXgHpJj%252BmI9D3gGJVbuVYvtpThO30evfDRvEK6eMY9oAa4fvgxnPTZfAz5t9mBO8aepd0ER8dteu8ChC6tWqfU3hf7XTm0zdofMvsj2yqyxWx9P%252BxbcH%252Fdz47ddCPtPcUKadereOHGjZQVgyKkLLLFZ855DvXwemW%252BYAVdCKZO9F%252F%252BKVXBQASDBbr8rkU38s3ozuSNZDk4%252FEoX7SO8%252FcMf4lWbWArC%252BL%252BH1aWnFxahpEiV16OMCshWQyQe1IF9JGqx5OEJCPnA7nievfNlN9IM%252FofLBX8Sz4LV3U0zV5hvIRxVeUEWnBBPQGL65Iqwpj0KcxrNGNb2W1lmGDLBZhXEkm1f0zXCFuq9amkT8uaOwj28iGoky7uAepM8jXyRr7%252FW8XsiYi0PhRULIJI%252BUKDeQZAbuNHjZnARzobYNhLmyi07E%252BbFOR5VGp%252FWHgjCpJw5eBpgGhGDaUaGzHIxI8s0rBMkDLhnjv1G1qm3keKPe%252FJH0gkqKTrQU%253D&SignatureMethod=HmacSHA1&SignatureVersion=4&Version=2013-10-15&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ334H7HOLAZGIHQA%2F20150714%2Feu-central-1%2Fec2%2Faws4_request&X-Amz-Date=20150714T091130Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=d6440df17a61954a4f670190d5909e7d4bbfaab5073dcd9434b058b1866b396f HTTP/1.1
> User-Agent: curl/7.35.0
> Host: ec2.eu-central-1.amazonaws.com
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Transfer-Encoding: chunked
< Date: Tue, 14 Jul 2015 09:11:30 GMT
* Server AmazonEC2 is not blacklisted
< Server: AmazonEC2
<
{ [data not shown]
* Connection #0 to host ec2.eu-central-1.amazonaws.com left intact
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+
mattheworiordan commented 9 years ago

@timkay Note I am running this command from an instance in eu-central-1 region.

timkay commented 9 years ago

I modified my ~.awssecret file to contain an invalid key, and I got

aws describe-tags --region eu-central-1 --AWS4

+-------------+--------------------------------------------------------------+ Code Message

+-------------+--------------------------------------------------------------+ | AuthFailure | AWS was not able to validate the provided access credentials | +-------------+--------------------------------------------------------------+

You do need new keys for each new region. Please take a look at the possibility that your keys are no good.

...Tim

On Tue, Jul 14, 2015 at 2:13 AM, mattheworiordan notifications@github.com wrote:

@timkay https://github.com/timkay Note I am running this command from an instance in eu-central-1 region.

— Reply to this email directly or view it on GitHub https://github.com/timkay/aws/issues/97#issuecomment-121174739.

Tim Kay m: +1-650-248-0123 Skype: timkay

mattheworiordan commented 9 years ago

I am only using IAM authentication though, and the command works with the Amazon aws CLI tool.

timkay commented 9 years ago

Yes, but the credentials are stored in a different place. Make sure you have valid credentials in ~/.awssecret On Jul 16, 2015 5:30 AM, "mattheworiordan" notifications@github.com wrote:

I am only using IAM authentication though, and the command works with the Amazon aws CLI tool.

— Reply to this email directly or view it on GitHub https://github.com/timkay/aws/issues/97#issuecomment-121943486.

mattheworiordan commented 9 years ago

Sorry @timkay I am not following you. We never store any credentials on the instance themselves and rely entirely on IAM

timurb commented 9 years ago

Do you mean IAM role attached to the instance?

mattheworiordan commented 9 years ago

Yes, we use CloudFormation and IAM assigned to the instance

Sent from my phone

On 16 Jul 2015, at 15:36, Timur Batyrshin notifications@github.com wrote:

Do you mean IAM role attached to the instance?

— Reply to this email directly or view it on GitHub.

timkay commented 9 years ago

I hadn't realized that you are using role-based authentication. In that case, you have to add --role. However, then we get a different error, which I am working on now.

benholtz commented 8 years ago

has something to do with this, but i'm not sure how to fix... http://aws.amazon.com/blogs/aws/aws-region-germany/

"For Developers – Signature Version 4 Support This new Region supports only Signature Version 4. If you have built applications with the AWS SDKs or the AWS Command Line Interface (CLI) and your API calls are being rejected, you should update to the newest SDK and CLI. To learn more, visit Using the AWS SDKs and Explorers."