search

aws / git-remote-codecommit

An implementation of Git Remote Helper that makes it easier to interact with AWS CodeCommit
Apache License 2.0
271 stars 38 forks source link

Using credential_process results in a 404 error #7

Open benbridts opened 4 years ago

benbridts commented 4 years ago

When using an external credential process get-remote-codecommit always returns a 404.

I'm using aws-vault to test this, but your credential_process could be a cat of a json file.

I'm using aws-vault to handle my credentials, this is my configuration:

# .aws/config

[profile codecommit]
region=eu-west-1
credential_process = aws-vault exec sandbox-profile --json

[profile tmp]
region=eu-west-1
output=json

I verified that aws-vault was working and the credentials where correct by using sts get-caller-identity and testing the aws-vault credential output in a temporary profile:

aws --profile codecommit sts get-caller-identity
{
    "UserId": "AROAIR7P4ZLYKGYCEVD6M:1583408935234570000",
    "Account": "XXXXXXXXXXX",
    "Arn": "arn:aws:sts::XXXXXXXXXXX:assumed-role/ROLE_NAME/1583408935234570000"
}

aws-vault exec sandbox-profile --json
{"Version":1,"AccessKeyId":"ASIAUBJLRPEKAZ2K7YC4","SecretAccessKey":"XXX","SessionToken":"YYY","Expiration":"2020-03-05T12:10:05Z"}

cat >> ~/.aws/credentails <<EOF
[tmp]
aws_access_key_id = ASIAUBJLRPEKAZ2K7YC4
aws_secret_access_key = XXX
aws_session_token = YYY

git clone codecommit::eu-west-1://tmp@dns-test dns-test-test
Cloning into 'dns-test-test'...
remote: Counting objects: 19, done.
Unpacking objects: 100% (19/19), done.

However, when I try to use the codecommit profile, I get the following error:

rm -rf dns-test-test/
git clone codecommit::eu-west-1://codecommit@dns-test dns-test-test
Cloning into 'dns-test-test'...
fatal: unable to access 'https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/dns-test/': The requested URL returned error: 403
aashwinkumaar commented 4 years ago

Can you test this with the latest version (1.13) ?

benbridts commented 4 years ago

I've tested this with version1.13 and I still have issues. I added some debugging and got strange behavior.

Test 1: CLI only

I created a script "credentials.sh" that looks like this (edit the paths if your tempdir is somewhere else) (+ make it executable and put it somewhere in your path)

#!/usr/bin/env bash
date >> /tmp/credentials.log
aws-vault exec PROFILE --json > /tmp/credentals.json

cat /tmp/credentals.json >> /tmp/credentials.log
echo '' >> /tmp/credentials.log
cat /tmp/credentals.json

I configured that script as the credential process:

[profile codecommit]
region=eu-west-1
credential_process = credentials.sh

Executing that with the CLI already gives me a strange output in the log (I get 2 calls logged instead of 1):

aws --profile codecommit sts get-caller-identity
{
    "UserId": "AROAJHGZ6YFDCSP67K3AE:1583681636663673000",
    "Account": "XXXXXXXXXXX",
    "Arn": "arn:aws:sts::XXXXXXSXXXXX:assumed-role/REDACTED/1583681636663673000"
}

cat /tmp/credentials.log
Sun Mar  8 16:33:55 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHHQWWKGEL","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:48:56Z"}
Sun Mar  8 16:33:56 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHKLYGO5FR","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:48:57Z"}

Test 2: Via git

I edited git_remote_codecommit/__init__.py and added the following code to the top of the sign method to get some debug output:

  sys.stderr.write(str({
    'access_key': credentials.access_key,
    'secret_key': credentials.secret_key,
    'token': credentials.token
  }))

If I keep the same configuration and a git command I get even more log lines:

> /tmp/credentials.log
git pull --ff-only
fatal: unable to access 'https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/cloudar.landing-zone/': The requested URL returned error: 403
{'access_key': 'ASIAVYZ4FOHHJCVKUZWK', 'secret_key': 'REDACTED', 'token': 'FwoGZXIvYXdzEPn//////////wEaDKTgcD/Mw/cTLi+BNCK7AdBmhuGumGrDWMOMppCP4ihQT2Y0jV6CdpuUM04gMTEY4O4+abUdxSmgRg1kPqvhzXCoWvOfPEA4uKCz8PNZ5E/lSZNX2goTBQuS79pYjZAFiEFGdD/fbFVOmcLc+2wBsOITiV9fe6v0rMMFcWJpNMozq6Ag0Ur2IDZh4a2ZFcStDviNKn60MU+9L2WTAH8UlJ6DiDVT5oOAXqCSInxJSotBJAH01ukXmN8Seiv9FGyjHanwWnr0v6EuErIok6OU8wUyLWZXpVQeb5F2YCLrmYXaGI5Z6Feqmb7THSqupmuFCCusfafnsIyDnDkMtRrqHw=='}

cat /tmp/credentials.log
Sun Mar  8 16:38:54 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHLIG3GT57","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:53:55Z"}
Sun Mar  8 16:38:55 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHEB3FB2UO","SecretAccessKey":"REDACTED","SessionToken":"OMMITED,"Expiration":"2020-03-08T15:53:55Z"}
Sun Mar  8 16:38:55 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHAW3BRVV5","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:53:56Z"}
Sun Mar  8 16:38:56 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHN532BMG2","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:53:57Z"}
Sun Mar  8 16:38:57 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHJCVKUZWK","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:53:58Z"}
Sun Mar  8 16:38:58 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHHKVSRALF","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:53:58Z"}
Sun Mar  8 16:38:58 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHOFPF5KU3","SecretAccessKey":"REDACTED","SessionToken":"FwoGZXIvYXdzEPn//////////wEaDKTgcD/Mw/cTLi+BNCK7AdBmhuGumGrDWMOMppCP4ihQT2Y0jV6CdpuUM04gMTEY4O4+abUdxSmgRg1kPqvhzXCoWvOfPEA4uKCz8PNZ5E/lSZNX2goTBQuS79pYjZAFiEFGdD/fbFVOmcLc+2wBsOITiV9fe6v0rMMFcWJpNMozq6Ag0Ur2IDZh4a2ZFcStDviNKn60MU+9L2WTAH8UlJ6DiDVT5oOAXqCSInxJSotBJAH01ukXmN8Seiv9FGyjHanwWnr0v6EuErIok6OU8wUyLWZXpVQeb5F2YCLrmYXaGI5Z6Feqmb7THSqupmuFCCusfafnsIyDnDkMtRrqHw==","Expiration":"2020-03-08T15:53:59Z"}
Sun Mar  8 16:38:59 CET 2020
{"Version":1,"AccessKeyId":"ASIAVYZ4FOHHIATEWRHQ","SecretAccessKey":"REDACTED","SessionToken":"OMMITED","Expiration":"2020-03-08T15:54:00Z"}

You'll notice that the session token used by git-remote-codecommit (from the call at 16:38:58) does not match the AccessKey (from the call at 16:38:57). The secret access key did match the session token in my test.

benbridts commented 4 years ago

Kyle did an investigation in https://github.com/aws/aws-cli/issues/5048. It looks like it's related to the refreshing of the credentials, as the problem goes completely away if the session is more than 15 minutes in the future.

changing

aws-vault exec PROFILE --json > /tmp/credentals.json

to

aws-vault exec PROFILE --json --assume-role-ttl=30m > /tmp/credentals.json

makes the problem go away