lionheart / git-bigstore

Bigstore is a Git extension that helps you track big files in your repositories.
Apache License 2.0
183 stars 28 forks source link

Update bigstore AWS credential requirements #46

Closed bkruger99 closed 5 years ago

bkruger99 commented 5 years ago

You can use this to assume a role, use a node role, or if you need to use keys or profile, just set your env vars. Backward compatibility is present but discouraged.


## Using an assumed role, all credential traces removed from .bigstore file.
bkruger$ export | grep AWS
declare -x AWS_PROFILE="dev_role"
declare -x AWS_SDK_LOAD_CONFIG="1"

bkruger$ grep -A4 "dev_role" ~/.aws/credentials
[profile dev_role]
region = us-west-2
role_arn = arn:aws:iam::XXXXXXX:role/XXXXXXX
source_profile = default

bkruger$ cat .bigstore
[bigstore]
    backend = s3
[bigstore "s3"]
    bucket = bigstore2350asdgfadsg

bkruger$ git bigstore init
Reading credentials from .bigstore configuration file.

bkruger$ git bigstore pull
pulling bigstore metadata...done
path/to/data/somefile.json  111111111
...

## Setting profile that uses aws keys.  Credentials removed from .bigstore file.
bkruger$ grep -A4 "dev_keys" ~/.aws/credentials
[profile dev_keys]
region = us-west-2
aws_access_key_id = redacted
aws_secret_access_key = redacted

bkruger$ export AWS_PROFILE="dev_keys"
bkruger$ git bigstore pull
pulling bigstore metadata...done
path/to/data/somefile.json  111111111
....

## Profile set in .bigstore file
bkruger$ export | grep AWS
declare -x AWS_SDK_LOAD_CONFIG="1"

bkruger$ cat .bigstore
[bigstore]
    backend = s3
[bigstore "s3"]
    bucket = bigstore2350asdgfadsg
    profile-name = dev_keys

bkruger$ git bigstore pull
pulling bigstore metadata...done
path/to/data/somefile.json  111111111
....

## Settings that have access key and secret key in the .bigstore profile:
Not tested.  This encourages bad behavior and potentially of checking keys into a git repo.
With AWS, we don't need this.
dlo commented 5 years ago

@bkruger99 This is awesome! Thanks so much for the contribution. Give me a day or so to review this but right off the bat it LGTM. 👍

bkruger99 commented 5 years ago

Cool. For those who are curious about the AWS_SDK_LOAD_CONFIG, it's something I that needs to be done for go-related software to do the same authentication items if I remember correctly.

bkruger99 commented 5 years ago

@dlo What can be done to help move this forward? Thanks! -b