Open LuisDeSiqueira opened 5 years ago
chrisroberts
commented
5 years ago Hi @LuisDeSiqueira. Thanks for the thorough issue report, it's very much appreciated :)
One followup question: Do you experience the same error if you provide your AWS configuration within the .sfn file only (and don't use the aws_profile_name setting)?
LuisDeSiqueira
commented
5 years ago Hello @chrisroberts,
Thank you for the prompt reply. If I do not provide aws_profile_name and instead obtain the value of the STS role by passing aws_sts_role_arn ENV['AWS_STS_ROLE_ARN'] in the configuration like so
# the sfn CLI. To view all available configuration
# options, please see:
# http://www.sparkleformation.io/docs/sfn/configuration.html
require_relative './lib/helpers/naming'
require_relative './lib/helpers/tagging'
require_relative './lib/helpers/getters'
require_relative './lib/helpers/raise_errors'
require_relative './lib/s3/bucket'
require_relative './lib/s3/bucket_names'
require_relative './lib/subnets/helpers'
require_relative './lib/sqs/queue'
require_relative './lib/sns/topic'
require_relative './lib/elasticsearch/metadata'
require_relative './lib/helpers/rspec'
Configuration.new do
apply_nesting 'deep'
processing true
options do
on_failure 'nothing'
notification_topics []
capabilities ['CAPABILITY_IAM','CAPABILITY_NAMED_IAM']
tags do
creator ENV['USER']
end
end
credentials do
provider :aws
#aws_access_key_id ENV['AWS_ACCESS_KEY_ID']
#aws_secret_access_key ENV['AWS_SECRET_ACCESS_KEY']
#aws_region ENV['AWS_REGION']
#aws_bucket_region ENV['AWS_REGION']
# or use default profile in ~/.aws/credentials
# works well with aws mfa https://jobvite.atlassian.net/wiki/display/PLAT/Set+aws+profile+credentials+with+mfa+token
#aws_profile_name ENV['AWS_PROFILE']
aws_sts_role_arn ENV['AWS_STS_ROLE_ARN']
end
endand call with ~/.rbenv/versions/2.4.1/gemsets/jobvite-sparkles/bin/sfn list --debug that it ignores the aws_sts_role_arn ENV['AWS_STS_ROLE_ARN'] completely and uses my default profile in ~/.aws/config to reach the incorrect non-STS account I default to.
if I put direct aws credentials for my default account in the SFN config then it will not go to profile and will use that configuration.
I think a crucial detail here is that I do not have regular credential access to the account that the STS role permits me access to. Therefore there is no way for me to run a test where I am authenticating regularly against that account.
pedrocarrico
commented
5 years ago @LuisDeSiqueira try to unset AWS_PROFILE in your terminal and try with just AWS_STS_ROLE_ARN
Worked for me, still I'd love to be able to use AWS_PROFILE and I'm seeing the same error as you are.
cc @chrisroberts
LuisDeSiqueira
commented
5 years ago @pedrocarrico
Thank you so much for the hint. It's true I have found that with a .sfn of
# This is an auto-generated configuration file for
# the sfn CLI. To view all available configuration
# options, please see:
# http://www.sparkleformation.io/docs/sfn/configuration.html
Configuration.new do
apply_nesting 'deep'
processing true
options do
on_failure 'nothing'
notification_topics []
capabilities ['CAPABILITY_IAM','CAPABILITY_NAMED_IAM']
tags do
creator ENV['USER']
end
end
credentials do
provider :aws
aws_sts_role_arn ENV['AWS_STS_ROLE_ARN']
end
endand nothing else in my environment related to AWS
❯ printenv|grep AWSand providing the STS var inline
❯ AWS_STS_ROLE_ARN=arn:aws:iam::REDACTED:role/qa_redshift_connector_administer sfn list
Name Created Updated Status Template Description
prod-udc 2018-11-16 06:23:02 UTC CREATE_COMPLETE redshift_connector.rb: Creates a redshift_connector stack for 'someone' i
n 'prod'
I get desirable results. So this issue is at least isolated to using the ~/.aws/config type settings.
cc @chrisroberts
Given a .sfn config of
and the following dependency set
and the following Gemfile which produces the above
The sfn list (errors with any command, list, create, etc) as executed like so
AWS_PROFILE=harishtest sfn list --debugwhere haristest is the name of my test profile indicating a role which otherwise works with the aws cli cloudformation interface.but here is the example config
The following error occurs
If I change the following file from the miasma-aws 0.3.22 gem: miasma-0.3.22/lib/miasma-aws/api.rb from it's original state to
Essentially, using aws-sdk-core to do the sts call and filling the correct information to Smash.new, then I do not see this failure, there is a call on "retryable_requests" that recursively loops.
I doubt the correct fix is shoehorning aws-sdk-core but this is as far as I've gotten so far. If I do this change and use aws-sdk-core then everything returns as expected from
like so
I know this is effecting the sfn gem and I will post there as well. I thought I would submit the issue here as well.
here is the issue there: https://github.com/sparkleformation/sfn/issues/284