SignatureDoesNotMatch error

[thomaswitt]

I keep on getting a A client error (SignatureDoesNotMatch) occurred when calling the ListUsers operation: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

I set the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION.

[bclodius]

I was facing this issue with the following scenario; both on rhel7 and ubuntu. I think it has something to do with non-alpha characters as others have mentioned

• s3 bucket protected for a single role
• ec2 instance with said role needs to assume same role before accessing protected s3 bucket

aws sts assume-role --role-arn <role_name> --role-session-name default --output json --query Credentials > credentials.json
export AWS_ACCESS_KEY_ID=`sed -n 's/.*"AccessKeyId": "\(.*\)"/\1/p' credentials.json`
export AWS_SECRET_ACCESS_KEY=`sed -n 's/.*"SecretAccessKey": "\(.*\)",/\1/p' credentials.json`
export AWS_SESSION_TOKEN=`sed -n 's/.*"SessionToken": "\(.*\)",/\1/p' credentials.json`

I believe something with the --query was messing up the credentials. When running the commands manually and copying and pasting the values; then setting the environment variables it seemed to work just fine.

[jesseHFG]

I had the same problem when running awscli version 1.10 on Mac (installed via pip) versus Ubuntu (Amazon image) awscli version 1.2.9. aws configure --profile user produces two different configurations under each. Version 1.10 produced two files: config and credentials. Version 1.2.9 just produced a config file (but with the credential information). When I deleted the credentials and config file produced by version 1.10 and copied the config file produced by version 1.2.9 everything worked, even with special characters and running awscli version 1.10. The config file produced by version 1.2.9 is

[profile FOO0]
aws_secret_access_key = FOO1
aws_access_key_id = FOO2
output = FOO3
region = FOO4

[schickling]

Can confirm that it's due to non-alphanumeric characters.

[nickfrandsen]

I have the same problem with a secret key containing a +. However I am not the owner of the S3 account and can't easily create a new one. Has anyone found any fix other than creating a new key without special characters?

[scottstensland]

tl;dr Solution : repeatedly re-recreate credentials UNTIL your aws_secret_access_key does NOT contain non-alphanumeric characters ... in my case aws_secret_access_key contained a + ( plus sign)

I just did a fresh install ... same issue ... this is on Ubuntu ... its not a local OS issue its an Amazon API issue so ignore other comments which say moving off OSX fixes it

aws ec2 describe-instances

An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials

------------------------------------------------

aws ec2 describe-security-groups

An error occurred (AuthFailure) when calling the DescribeSecurityGroups operation: AWS was not able to validate the provided access credentials

------------------------------------------------

aws ecr get-login

An error occurred (InvalidSignatureException) when calling the GetAuthorizationToken operation: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been 'POST /

content-type:application/x-amz-json-1.1 host:ecr.us-east-1.amazonaws.com x-amz-date:20160615T182955Z x-amz-target:AmazonEC2ContainerRegistry_V20150921.GetAuthorizationToken

content-type;host;x-amz-date;x-amz-target 44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a'

The String-to-Sign should have been 'AWS4-HMAC-SHA256 20160615T182955Z 20160615/us-east-1/ecr/aws4_request 86c2e3c850acd6765a1ca6aa626c6e9a6c85284f3954f45e170f51b5b89961c7'

------------------------------------------------

aws iam list-users

An error occurred (SignatureDoesNotMatch) when calling the ListUsers operation: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been 'POST /

host:iam.amazonaws.com x-amz-date:20160615T183516Z

host;x-amz-date b6359072c78d70ebee1e81adcbab4f01bf2c23245fa365ef83fe8f1f955085e2'

The String-to-Sign should have been 'AWS4-HMAC-SHA256 20160615T183516Z 20160615/us-east-1/iam/aws4_request ad9f7581f0bf25ecae56355a6c96b60f3dc3188efbbdb3d0d4806e9f2c5cf8a9'

------------------------------------------------

aws --version aws-cli/1.10.38 Python/2.7.11+ Linux/4.4.0-22-generic botocore/1.4.28

------------------------------------------------

cat /root/.aws/credentials [default] aws_access_key_id = AKIAJ7FCEUVVSGX7KZGQ aws_secret_access_key = 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

    Notice the plus sign ( + ) in above aws_secret_access_key   !!!!

   aws only works when aws_secret_access_key does NOT contain non-alpha chars

------------------------------------------------

SOLUTION : After deleting and creating fresh credentials where aws_secret_access_key did NOT have plus sign ( + ) all was well above commands started to work OK

[bitmuncher]

I had the same problem. Re-creating credentials until I had no special characters in it worked for me.

[jnewbigin]

I found that when I copied & pasted the credentials they had ^M characters on the end which was causing the error.

[floelhoeffel]

Getting a secret key without + fixed it for me too...

[jghaines]

Note - if you are seeing this issue under (boot2docker VM'd) docker, it can be that the VM clock is out of sync. See: http://stackoverflow.com/questions/24551592/how-to-make-sure-dockers-time-syncs-with-that-of-the-host

[admix]

I'm having the same issue. What if I don't have permissions to regenerate the credentials.. any possible fixes using other ways ?

[james-gonzalez]

UPDATE: I fixed this by running rm -rf .aws/cli/cache/

I'm having this issue now as well. When attempting to assume role

Version: aws-cli/1.11.17 Python/2.7.10 Darwin/16.1.0 botocore/1.4.74

Edit: Tried also updating again now, same problem: aws-cli/1.11.18 Python/2.7.12 Darwin/16.1.0 botocore/1.4.75

Output:

Assuming role arn:aws:iam::XXXXXXXX:role/XXXX-staging using profile default

An error occurred (SignatureDoesNotMatch) when calling the AssumeRole operation: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/

host:sts.amazonaws.com
x-amz-date:20161118T102600Z

host;x-amz-date
41db88384d54dc0783e616aa0662ebffce8832b35025195052029a5dc0e33c0e'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20161118T102600Z
20161118/us-east-1/sts/aws4_request
786b3d624f5aeea9ffcb2b802b177a4c2aebbfed608a2464ee684c6972bc6bc6'

[mpopova-yottaa]

The same issue still exists with the latest (up to date) version of AWS CLI as well. I've just upgraded my 1.8.3 CLI to 1.11.19 - and was not able to execute any commands with a new user/keys I've created. Had to re-cycle about 5 keys before I got a pair where the Secret key did not have any non-alphabetical characters. Once I stumbled upon such pair - CLI works fine.

Very annoying - I wish Amazon would not be generating invalid keys that they cannot process.....

[james-gonzalez]

@mpopova-yottaa did you try completely clearing the awe-cli cache ? Try deleting the entire ~/.aws directory (make a copy of your config/credential files)

[aebrow4]

aws ec2 describe-instances runs for me, but when trying to create cloud formation stack:

The Canonical String for this request should have been
'POST
/

amz-sdk-invocation-id:18d13b66-80ae-f676-c0cf-dbf875edb1aa
amz-sdk-retry:3/345/470
host:cloudformation.us-west-1.amazonaws.com
user-agent:aws-sdk-java/1.11.20 Mac_OS_X/10.10.5 Java_HotSpot(TM)_64-Bit_Server_VM/25.91-b14/1.8.0_91
x-amz-date:20161127T194448Z

amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date
aca0973fb4ac4029ec038d9c80b4afa23b6d305822b10e6bc32751ee1bd311d5'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20161127T194448Z
20161127/us-west-1/cloudformation/aws4_request
cb0286a8727afcc465171ab991accde0aaa7210e160a9ba3196e2a5e4305f428' (Service: AmazonCloudFormation; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: f52abbd4-b4d9-11e6-b989-d5c3d50d5e7b)

config details:

$: aws --version >> aws-cli/1.11.21 Python/2.7.12 Darwin/14.5.0 botocore/1.4.78

$: aws configure list >>

      ----                   -----                  ----        --------
profile                    not set>             None    None
access_key     ****************RTSA                env
secret_key     ****************UC3r                  env
region                us-west-1              env       AWS_DEFAULT_REGION

The secret key only contains alphanumerics, so I am stuck.

[james-gonzalez]

@aebrow4 It's down the cache for awe-cli. Try deleting: .aws/cli/cache/

[aebrow4]

@cultavix there is no cli directory inside .aws

[saraedum]

I got this error when running aws glacier upload-archive with --archive-description "`date`". When I use --archive-description "`date +%Y/%m/%d`" it works fine, so there seems to be some escaping issue.

[alastair-gatpark]

I was having a similar issue:

λ aws s3 sync s3://foo-bar-assets/ . --exclude "*/*.mp4" --exclude "*.mp4" fatal error: An error occurred (SignatureDoesNotMatch) when calling the ListObjects operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I tried syncing the time with an NTP server (successfully), but this did not resolve the problem. Regenerating keys until I had a set with no special characters fixed it.

λ aws --version aws-cli/1.11.16 Python/2.7.9 Windows/8 botocore/1.4.73

[houssemFat]

I have the same issue using awscli and sample python code (boto3). Note that i'm using IBM object storage S3 api compatible, I can list buckets and their contents but can't upload (both for pyhton code and cli). I tested the scenario with the ruby aws-sdk and it works fine (upload/download). -- configuration aws-cli/1.2.9 Python/3.4.3 Linux/3.19.0-33-generic

[samato88]

Just started having this same issue with a script I've been using for months for depositing multi part uploads into Glacier. Can still deposit fine via aws cli, so credentials still work, but script using boto3 fails with:

"botocore.exceptions.ClientError: An error occurred (InvalidSignatureException) when calling the InitiateMultipartUpload operation: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."

aws --version is aws-cli/1.11.38 Python/2.7.10 Darwin/15.6.0 botocore/1.5.1

(yes I updated everything thinking that might fix the problem...no such luck.)

[trushkevich]

getting a new pair of keys without any special characters (it was + in my case) fixed the issue for me

[ewdurbin]

another confirmation here of incorrect handling of + in secret key.

[jamesls]

Hi everyone, here's a script that I've used to try and repro this issue: https://gist.github.com/jamesls/00ef7fcc0ac39ba8b06956d165c42f6d . This is what the script does:

• Creates a new access_key/secret_key pair via aws iam create-access-key in a loop until it finds credentials that have either a + or a / char.
• It creates a new "testcreds" profile with these newly generated credentials
• It tries to make invoke various AWS CLI commands to ensure these credentials work

It does this in an infinite loop until an API call fails. So far I haven't been able to get it to fail. Secret keys that have + and / are working for me.

At this point we've confirmed that it's definitely possible to use secret_keys that have a + or / so I don't think the root cause is something as straightforward as a bug in our signing code that breaks when + is present. Rereading the comments in this thread, it might be an issue with how credentials are being entered into the ~/.aws/config or ~/.aws/credentials file. Perhaps there's some platform specific thing that's altering characters that contain + or /.

So for anyone that's still running into this issue, it'd be really helpful if you can provide some additional information:

1. How are you getting credentials (copy/paste from console, aws iam create-access-key, CSV file from console, etc)?
2. How are you configuring the CLI to use these credentials? Are you running aws configure and entering the values when prompted? Are you doing this programmatically by running aws configure set? Are you manually editing ~/.aws/config and/or ~/.aws/credentials? Setting the various AWS_* env vars?

Bonus things that would be even more helpful if possible:

3. If you use other SDKs, do these credentials that don't work in the CLI work in other AWS SDKs?
4. If you have a test account or test IAM user, does running the script I'm using to test ever fail for you?

Any additional info that might help us repro this issue on our end would be great.

[samato88]

I am still having this issue. To answer your questions: Created credentials in amazon console and cut/paste into ~/.aws/credentials (edited with emacs on a mac)

From the troubleshooting I've done so far (and I'm a novice at this...) the 'Canonical String' is correct, but the 'String-to-Sign' is wrong, having a different last line. I.e. when I print the return value of auth.py string_to_sign, the number produced from 'sha256(canonical_request.encode('utf-8')).hexdigest())' is different than the number reported in the returned error "The String-to-Sign should have been" .

There are no special characters in my credentials, and the do work when using other tools, e.g. CrossFTP (which I don't want to use!!!)

Any insights would be greatly appreciated!!

[jamesls]

@samato88 that appears to be a different issue there. If you could share any debug info (making sure to remove any sensitive information) that would help.

The string_to_sign is not influenced by the secret_access_key. When we sign a request, we take the HTTP request we're about to send, convert it to a string (i.e the string ot sign), and then using the secret key, we sign that string with the secret key (skipping a few details here). So any issues with the string to sign would be separate from this issue.

Could you open another issue and provide the --debug output (or include the complete request and error message from the service)? We'd be happy to look into this for you.

[samato88]

Thanks jamesls - I've opened a separate issue at: https://github.com/aws/aws-cli/issues/2474

Any insights much appreciated.

[AlexParra03]

If your system time is off by more than 5 minutes, you will not be able to use the CLI

just run... sudo ntpdate -s time.nist.gov

then try again

[robotzero]

I had the same issue, my secret key contains "+" sign. I've deleted my .aws/credentials file and re-run aws configure. After that my query to sqs queue to receive message worked.

[kcmanoj]

Thanks @AlexParra03 . I had same issue and your comments helped me to resolve.... :-)

[jamesls]

@robotzero do you recall how you entered your credentials? Did you run aws configure and then copy/paste the values from the web console?

[robotzero]

Yes, I ran aws configure and copy pasted the values.

[damienrj]

Yes, I had a + in my secret, which caused this error still. Making new keys without special characters fixed the problem.

[shwetapurushe]

aws --version aws-cli/1.11.78 Python/3.6.1 Darwin/15.6.0 botocore/1.5.41

Had a + and a / in my secret. Secrets without these solved the problem for me.

[adityanatraj]

Also had this issue.

$ aws --version
aws-cli/1.11.44 Python/3.5.3 Linux/4.10.0-19-generic botocore/1.5.7

$ lsb_release -sd 
Ubuntu 17.04

Had a "+" in the credentials. Was resolved by regenerating access key without them. As a note: "/" is a fine character to have.

[jamesls]

@adityanatraj @shwetapurushe @damienrj can you all answer these questions here? At this point we're trying to get more information about your environment and how you're entering credentials. As mentioned in that comment, generating a secret key with + doesn't repro the issue for us, so it's possibly related to a combination of your environment and/or how you're entering credentials.

In other words, we need more information from people so can troubleshoot what's going on.

[adityanatraj]

@jamesls

1. How did I get the credentials?

I got the credentials by copying them from the web console.

2. How do I configure CLI to use them?

I manually created the two files:

~/.aws/config

[default]
region = us-east-1
output = json

~/.aws/credentials

[default]
aws_access_key_id = ACCESS_KEY_HERE 
aws_secret_access_key = SECRET_ACCESS_KEY_THAT_BREAKS_WITH_A_+_SIGN

Sorry, I can't help with the bonus questions as I've purged my access keys containing '+' signs, so the problem doesn't show up.

[jamesls]

@adityanatraj thanks, every bit helps.

The next step that would help us troubleshoot is to figure out if this is an issue with just the CLI or an issue with the credentials themselves. For anyone that's still running into this issue, it would be really helpful if you could try these credentials in other AWS SDKs. To help with this, I've put a sample repo/script together that makes this easy if you don't want to set this up yourself: https://github.com/jamesls/aws-creds-test . Just clone the repo and run make install, make test.

/tmp $ mkdir /tmp/repro-cli-602
/tmp $ cd /tmp/repro-cli-602/
/tmp/repro-cli-602 $ git clone git://github.com/jamesls/aws-creds-test
Cloning into 'aws-creds-test'...
...
/tmp/repro-cli-602 $ cd aws-creds-test/
/tmp/repro-cli-602/aws-creds-test (master u=) $ make install
npm install
aws-js-cli@1.0.0 /private/tmp/repro-cli-602/aws-creds-test
├─┬ aws-sdk@2.45.0
...
pip install -r requirements.txt
Requirement already satisfied: botocore<2.0.0,>=1.5.0 in /usr/local/lib/python2.7/site-packages (from -r requirements.txt (line 1))
...

/tmp/repro-cli-602/aws-creds-test (master u=) $ make test
./test-creds.sh
Testing python...
Access Key:
Secret Access Key:
AKID   hash: 4e7c36343646e1fa7495092bffcd4b9b7dd00f2f5014a189ab81f326e6472a62
AKID length: 20

SAK    hash: 941a655993caccb1a1218883b97a88b6f41762c6d03902f1cdd1e2a5de5fd82e
SAK  length: 40
Successfuly made an AWS request with the provided credentials.

Testing javasript...
Access Key: ********************
Secret Access Key: ****************************************
AKID   hash: 4e7c36343646e1fa7495092bffcd4b9b7dd00f2f5014a189ab81f326e6472a62
AKID length: 20

SAK    hash: 941a655993caccb1a1218883b97a88b6f41762c6d03902f1cdd1e2a5de5fd82e
SAK  length: 40
Sucessfully made an AWS request with the provided credentials.

For people running into this issue, please run the test script and share the output.

[gilad-feelter]

This same exception happened to me on a PutObject request (C#), whenever the object metadata had Unicode characters.

[gchiu]

I had the same issue. New secret keys with + and / don't work. I generated new keys without those characters and they work fine. Your test script is for linux and I'm running windows. I pasted my credentials using Control-C and Control-V using windows shell and aws configure and I was only using aws cp

[LewisLebentz]

Tested this too, and it works fine as long as the secret key has no symbols, as mentioned above.

Hope to see this resolved soon, it's a pain having to keep regenerating credentials until I get one that works!

[LewisLebentz]

I raised a support ticket to AWS yesterday, and today it appears to be resolved

I've tested multiple times and + and / both now seem to work? I can no longer reproduce this bug.

[bonescs]

I had the same issue on my Pi. With the newest awscli (aws-cli/1.11.85 Python/3.4.2 Linux/4.9.24-v7+ botocore/1.5.48) I still had the issue:

root@pi:~# aws s3 ls s3:// An error occurred (SignatureDoesNotMatch) when calling the ListBuckets operation: The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.

Even with a secret key that had no special characters (+ or /) the access didn't work. The time was always in sync, unfortunately this wasn't the issue either.

In the ".aws/config" file I added a valid region (just for testing..) and suddenly it worked. Since I'm using compatible s3 storage (not s3 from Amazon)

[default] aws_secret_access_key = MYKEY aws_access_key_id = MYID region = us-east-1 <-- there was a "dummy" value before.

As it looks, the region must have a "correct" value. When I change it back to something different like "dummyvalue" I get the same error as mentioned above. Now with the value "us-east-1" it works! Hope this helps someone.

[eikenb]

I just ran into this as well. Also seems to be an issue with a '+' in the secret key. If I have the creds in environment variables I get the error. If I put them in the ~/.aws/credentials file (by editing by hand) I don't get the error.

[edit] Just noticed the environment variables were in a file formatted for dos (ff=dos in vim). It was this because I had just taken the .csv file as downloaded and edited it to change the entries into environment variables. When I re-formatted the file into 'ff=unix' I no longer got the error. The only difference between the 2 is the line return, dos uses "CR-NL" while unix is just "NL". My guess is that it was actually that "CR" character that was the problem.

[existeundelta]

me too, and also confirm the "+" issue

[tylerjharden]

If you run into this when using Docker for Mac, simply restarting Docker will fix the system time discrepancy.

[derchrisuk]

I was facing the same problem. Checked the secret, and it had +/ in it. Had to create a new id pair without a special character to get it to work

[boutell]

Creating new keypairs until I got one with no special characters resolved it for me too; special characters (specifically +) simply do not work in ~/.aws/credentials.

[Checkroth]

Generated key without special characters (specifically + ) fixed the issue for me too.

Formatting file as per @eikenb 's comment does the trick as well.

[KumarAtwindsorinfosys]

Deleting the Key and creating new one worked for me.

[zgr024]

Just received this error. Updated system time which appeared to be more than 6 minutes off GMT. Fixed the issue immediately.