How to configure the X-Amz-Security-Token / AWS_SECURITY_TOKEN?

philipp-classen commented 2 years ago

Thanks for creating the library! It works, but I ran into two issues. The first is merely of an observation, while the second is a question about the implementation of X-Amz-Security-Token, which looks incomplete.

What I was trying to do is to list buckets like aws s3 ls. I'm using aws-vault and have the credentials available as environment variables.

I was assuming the client would automatically default to the variables from the environment, as many libraries do. Have when setting them manually, I also ran into a problem. Here is what I tried:

require "awscr-s3"

client = Awscr::S3::Client.new(ENV["AWS_REGION"], ENV["AWS_ACCESS_KEY_ID"], ENV["AWS_SECRET_ACCESS_KEY"])
puts client.list_buckets.buckets

It turns out that aws-vaul is generating temporary credentials: there is a AWS_SECURITY_TOKEN which has to be passed in the X-Amz-Security-Token header. When I tested with non-temporary credentials, the code above worked.

Regarding the security token, I did find it being implemented in the awscr-signer library in the Awscr::Signer::Signers::V4 class. But from awscr-s3, I did not see a way to pass the value. Thus, it is always nil, which explains why AWS fails with:

Unhandled exception: The AWS Access Key Id you provided does not exist in our records. (Awscr::S3::InvalidAccessKeyId)
  from lib/awscr-s3/src/awscr-s3/http.cr:114:9 in 'handle_response!'
  from lib/awscr-s3/src/awscr-s3/http.cr:88:16 in 'exec'
  from lib/awscr-s3/src/awscr-s3/http.cr:83:13 in 'exec:headers'
  from lib/awscr-s3/src/awscr-s3/http.cr:66:7 in 'get'
  from lib/awscr-s3/src/awscr-s3/client.cr:46:7 in 'list_buckets'
  from src/crystal-s3-hallo-world.cr:4:6 in '__crystal_main'

When I change the code in line 17 from

        def initialize(... @amz_security_token : String? = nil])

to

        def initialize(... @amz_security_token : String? = ENV["AWS_SECURITY_TOKEN"])

it works.

My questions:

What it make sense to create a constructor for the client where you don't have to pass the key and secret, but which will automatically use the standard environment variables as the fallback?
Am I missing a way to configure the security token? Or is that part not fully implemented yet?

I'm using Crystal 1.4.1 and my shard.lock looks like that:

version: 2.0
shards:
  awscr-s3:
    git: https://github.com/taylorfinnell/awscr-s3.git
    version: 0.8.3

  awscr-signer:
    git: https://github.com/taylorfinnell/awscr-signer.git
    version: 0.8.2

The environment is setup that I get from aws-vault looks like that (I think, the name of the environment variables is pretty standard):

AWS_DEFAULT_REGION=us-east-1
AWS_REGION=us-east-1
AWS_SESSION_EXPIRATION=2022-05-06T16:50:53Z
AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXuXXXXXXXXXXXXXXXXXXXXX
AWS_ACCESS_KEY_ID=XXXXXXXXTXXXXXXXXXXX
AWS_SESSION_TOKEN=IQo..xXxXxxxXxXxXXXxxxxxxxxxxxXxXXXxXXXxxxXxXXXXXXXXXXxxxxxXxxxxXxxxXXXXXXXxxxXXXxXxxXXxXXxXXxxXXxXXxxXxxxXxxXxxxXxxXXXxXxxxxxXxxXxxxXxxxXXxxxXxxXXXxxxxxxxxxxxxXXXXXxxxXXXxXXxxXXXxXXXxXXXXXxXxxXxXxxxxxXxxXxxXxXxxxxxxXXXxxXxxxXxxxxxxXXxXxXXXxxXXxxXXxxXxxxXxXxxxxxXxXxxXxXxXxXXXxXxxXxXXxXxXxXXxXxXxxxxxxXXXxXXxxxXxxxXXXxxXxxxXxxxXXxXxXXxxxxxxxXXXXXxxXxxxxxxxXxXXxxXxxXXxxXxXxxxXxXXxXXxXXxXXXXXxxxxXXXxxxXXxxXXXxXXxxxXXXxxxXxXXxxXxXxXXxxxXxXxxxXxxXXXxxxxxxXxxxxxxXXXxXXXxxxxxxXXxXXXxxxXxXXxxXxXXxxxxxXxxXXXxxxXXxxXxxxxxXXxxXxXXXXxXxxxXxXxXxxxxxxxXxXxxXXXxXxxxXXXxxXXxXxXXXxxXxXXXXxxXXxxXXXXxxXxXXxxxxxXXXxxxXxXXxXxXxxXxxxXXxxXxxxXXxXxxxXxxxxXxxxXxXXXXxXxXxxxxXxxxxxXXXXXXXXxxxXxXxXxXxxxxXXXxxxXXXxxxXXXXXxxxXXxXxXxXXXXXxxxxxXxXXxXXXXXxXXxxXxXxxxxxxXxXxxxxxxxXxXXxxxxxXxxxxxxxxxxxxxxxxXXXXXxxxXXxxxxXxxXxXxxXxXxxXxXXXxxxxxxXxXxxXXxxxxxxXxXxxxxxxXXxxxXxxxXxxXxXxxxxxXxXxxxXXXXxXxxXXxXXXxXxxXxXxXXxxxXxxxxXxXxxxxXXxxxxxxxxXZQ=
AWS_SECURITY_TOKEN=IQo..xXxXxxxXxXxXXXxxxxxxxxxxxXxXXXxXXXxxxXxXXXXXXXXXXxxxxxXxxxxXxxxXXXXXXXxxxXXXxXxxXXxXXxXXxxXXxXXxxXxxxXxxXxxxXxxXXXxXxxxxxXxxXxxxXxxxXXxxxXxxXXXxxxxxxxxxxxxXXXXXxxxXXXxXXxxXXXxXXXxXXXXXxXxxXxXxxxxxXxxXxxXxXxxxxxxXXXxxXxxxXxxxxxxXXxXxXXXxxXXxxXXxxXxxxXxXxxxxxXxXxxXxXxXxXXXxXxxXxXXxXxXxXXxXxXxxxxxxXXXxXXxxxXxxxXXXxxXxxxXxxxXXxXxXXxxxxxxxXXXXXxxXxxxxxxxXxXXxxXxxXXxxXxXxxxXxXXxXXxXXxXXXXXxxxxXXXxxxXXxxXXXxXXxxxXXXxxxXxXXxxXxXxXXxxxXxXxxxXxxXXXxxxxxxXxxxxxxXXXxXXXxxxxxxXXxXXXxxxXxXXxxXxXXxxxxxXxxXXXxxxXXxxXxxxxxXXxxXxXXXXxXxxxXxXxXxxxxxxxXxXxxXXXxXxxxXXXxxXXxXxXXXxxXxXXXXxxXXxxXXXXxxXxXXxxxxxXXXxxxXxXXxXxXxxXxxxXXxxXxxxXXxXxxxXxxxxXxxxXxXXXXxXxXxxxxXxxxxxXXXXXXXXxxxXxXxXxXxxxxXXXxxxXXXxxxXXXXXxxxXXxXxXxXXXXXxxxxxXxXXxXXXXXxXXxxXxXxxxxxxXxXxxxxxxxXxXXxxxxxXxxxxxxxxxxxxxxxxXXXXXxxxXXxxxxXxxXxXxxXxXxxXxXXXxxxxxxXxXxxXXxxxxxxXxXxxxxxxXXxxxXxxxXxxXxXxxxxxXxXxxxXXXXxXxxXXxXXXxXxxXxXxXXxxxXxxxxXxXxxxxXXxxxxxxxxXZQ=

taylorfinnell commented 2 years ago

Hi, yes, I think it's reasonable for that constructor to change and for the defaults to be the ENV vars, if set. Happy to merge a PR for that.

You are correct, as it stands now I don't think any of the config will auto configure off the ENV

Thanks!

anton7c3 commented 5 months ago

There's a third party shard to handle the process of obtaining the credentials: https://github.com/y2k2mt/aws-credentials.cr With the credentials received, the remaining part in this shard is pretty simple: https://github.com/taylorfinnell/awscr-s3/pull/114

taylorfinnell / awscr-s3

How to configure the X-Amz-Security-Token / AWS_SECURITY_TOKEN? #105