search

eksctl-io / eksctl

The official CLI for Amazon EKS
https://eksctl.io
Other
4.93k stars 1.41k forks source link

cannot get role ARN for current session #1408

Closed 7ynk3r closed 4 years ago

7ynk3r commented 5 years ago

I'm getting started with EKS and I'm following Amazon guide https://docs.aws.amazon.com/en_pv/eks/latest/userguide/getting-started-eksctl.html

I've defined an Administrator user under the Administrators group with the following policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "*",
      "Resource": "*"
    }
  ]
}

and with it I've set 

$ cat ~/.aws/credentials                                                                                                                               
[default]
aws_access_key_id = ___
aws_secret_access_key = ___

$ cat ~/.aws/config                                                                                                                                    
[default]
region = us-west-2
output = json

I'm able to 

$ aws eks list-clusters
{
    "clusters": []
}

$ aws sts get-caller-identity                                                                                                                          {
    "UserId": "___",
    "Account": "___",
    "Arn": "arn:aws:iam::___:user/Administrator"
}

but the following fails

$ eksctl create cluster \                                                                                                                              --name prod \      
--version 1.14 \
--nodegroup-name standard-workers \
--node-type t3.medium \
--nodes 3 \
--nodes-min 1 \
--nodes-max 4 \
--node-ami auto
[ℹ]  using region us-west-2
[✖]  checking AWS STS access – cannot get role ARN for current session: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: net/http: TLS handshake timeout

$ eksctl get cluster
2019-10-06T22:58:51-07:00 [✖]  checking AWS STS access – cannot get role ARN for current session: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: net/http: TLS handshake timeout

What am I missing?

ballu510 commented 5 years ago

Hi 7yn3r, This issue has been addressed here - https://github.com/weaveworks/eksctl/issues/57. By the way I tried you command that you posted, it works for me. You may want name your EKS cluster with the parameter --name Hope this helps.

nrdlngr commented 5 years ago

@7ynk3r, I wrote the guide you referenced, so I'm interested in this issue because I've seen a few other EKS doc customers have this same STS timeout and I haven't been able to root cause it yet. You are not alone.

@ballu510 can you be more specific about what in that issue applies to this one? It's hard to tell.

Also, eksctl will happily name a cluster if a name is not provided, so I doubt that is related.

eksctl should use the same credential chain as the AWS CLI, so if the AWS CLI works, so should eksctl.

ballu510 commented 5 years ago

Hi @7yn3r -

I simply created an IAM user with userName - eksadmin and granted Administrator access as you did and created cluster without any issue.

Next, set credentials:

[eksadmin] aws_access_key_id = ---- aws_secret_access_key = ---

Config: [eksadmin] region = us-west-2

Finally, ran below script with the profile set to eksadmin: eksctl create cluster --name prod-xxxxx1 --version 1.14 --nodegroup-name standard-workers --node-type t3.medium --nodes 3 --nodes-min 1 --nodes-max 4 --node-ami auto --profile eksadmin

[ℹ] eksctl version 0.7.0 [ℹ] using region us-west-2

[✔] EKS cluster "prod-xxxxx1" in "us-west-2" region is ready

You can confirm the user assigned to the cluster under ~/.kube/config -

7ynk3r commented 5 years ago

Hi @ballu510,

I've tried creating a brand new admin user, added it to a new profile on ~/.aws/credentials and then tried running your command but the result is the same.

I have no directory ~/.kube on my local.

About #57, looks similar but it's not the same issue. In that case they're able to create a cluster but not with a particular profile.

Are you using a brand new AWS account?

Do you have any roles created? I have only the followings image

Thanks for taking the time to look at this issue, I really appreciate it.

ballu510 commented 5 years ago

Hi @7ynk3r,

Ok - I created a new brand new account, created a user "testing" and assigned Admin access. aws sts get-caller-identity { "Account": "893xxxxxxxxx", "UserId": "AIDA47XXXXXXXXXX", "Arn": "arn:aws:iam::893xxxxxxxxx:user/testing"

eksctl create cluster --name backup --version 1.14 --nodegroup-name standard-workers --node-type t3.medium --nodes 3 --nodes-min 1 --nodes-max 4 --node-ami auto [ℹ] eksctl version 0.7.0 [ℹ] using region us-west-2 [ℹ] setting availability zones to [us-west-2b us-west-2c us-west-2a]

[✔] EKS cluster "backup" in "us-west-2" region is ready

It created ./kube/config file for me. name: backup.us-west-2.eksctl.io contexts:

Unable to create an issue that you are encountering.

cheers

7ynk3r commented 5 years ago

Hi @ballu510,

I've reviewed all the steps once again, I just created a brand new AWS account and the result is still the same...

eksctl create cluster --name backup --version 1.14 --nodegroup-name standard-workers --node-type t3.medium --nodes 3 --nodes-min 1 --nodes-max 4 --node-ami auto
[ℹ]  eksctl version 0.7.0
[ℹ]  using region us-west-2
[✖]  checking AWS STS access – cannot get role ARN for current session: RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: net/http: TLS handshake timeout
$ aws sts get-caller-identity                                                                                                                       
{
    "UserId": "AIDA...",
    "Account": "7624....",
    "Arn": "arn:aws:iam::7624....:user/admin"
}
$ aws --version
aws-cli/1.16.260 Python/3.7.4 Darwin/18.2.0 botocore/1.12.250
$ pip3 list
Package            Version  
------------------ ---------
appnope            0.1.0    
astroid            2.0.4    
autopep8           1.4.3    
awscli             1.16.260 
backcall           0.1.0    
bleach             3.0.2    
botocore           1.12.250 
certifi            2018.8.24
chardet            3.0.4    
colorama           0.4.1    
decorator          4.3.0    
defusedxml         0.5.0    
docutils           0.15.2   
entrypoints        0.2.3    
future             0.17.1   
gym                0.10.9   
idna               2.6      
ipykernel          5.1.0    
ipython            7.1.1    
ipython-genutils   0.2.0    
ipywidgets         7.4.2    
isort              4.3.4    
jedi               0.13.1   
Jinja2             2.10     
jmespath           0.9.4    
jsonschema         2.6.0    
jupyter            1.0.0    
jupyter-client     5.2.3    
jupyter-console    6.0.0    
jupyter-core       4.4.0    
lazy-object-proxy  1.3.1    
MarkupSafe         1.1.0    
mccabe             0.6.1    
mistune            0.8.4    
nbconvert          5.4.0    
nbformat           4.4.0    
notebook           5.7.2    
numpy              1.15.4   
pandas             0.23.4   
pandocfilters      1.4.2    
parso              0.3.1    
pexpect            4.6.0    
pickleshare        0.7.5    
pip                19.1.1   
prometheus-client  0.4.2    
prompt-toolkit     2.0.7    
ptyprocess         0.6.0    
pyasn1             0.4.7    
pycodestyle        2.4.0    
pyglet             1.3.2    
Pygments           2.2.0    
pylint             2.1.1    
python-dateutil    2.7.5    
pytz               2018.7   
PyYAML             5.1.2    
pyzmq              18.0.1   
qtconsole          4.4.3    
requests           2.18.4   
Robinhood          1.0.1    
rsa                3.4.2    
s3transfer         0.2.1    
scipy              1.2.0    
selenium           3.141.0  
Send2Trash         1.5.0    
setuptools         41.0.1   
six                1.11.0   
terminado          0.8.1    
testpath           0.4.2    
tornado            5.1.1    
traitlets          4.3.2    
urllib3            1.22     
wcwidth            0.1.7    
webencodings       0.5.1    
wheel              0.33.4   
widgetsnbextension 3.4.2    
wrapt              1.10.11  
yolk3k             0.9

I'm using macOS Mojave 10.14.1

Let me know if you know of anything else I can try.

golgoth31 commented 5 years ago

Hi I'm having the same problem. Any ideas of what could happen ? The eksctl command with -v option at 5 gives me such output in the roleArn parameter : Action=AssumeRole&DurationSeconds=1800&RoleArn=arn%!!(MISSING)A(MISSING)aws%!!(MISSING)A(MISSING)iam%!!(MISSING)A(MISSING)%!!(MISSING)A(MISSING)

Is there a parsing failure of the ":" sign ?

golgoth31 commented 5 years ago

I found a workaround, I have to double quote the first ":" of the arn .... I can't understand why in my .aws/config file, my profile had a line like this: role_arn = arn:aws:iam::.... I change for that: role_arn = arn":"aws:iam::...... and the eksctl is ok

toahae commented 4 years ago

I don't why we have to set these environment variables to fix this error: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

bluecough commented 4 years ago

@golgoth31 your workaround seems to work for me. I am running Ubuntu 19.10 with eksctl version: [ℹ] version.Info{BuiltAt:"", GitCommit:"", GitTag:"0.11.1"}

I have to remove the workaround after I create a cluster. Then everything just works normally.

michaelbeaumont commented 4 years ago

@7ynk3r @bluecough @golgoth31 I'd love to know if this is still a problem for you. It looks like this is likely an issue with aws-sdk-go/the AWS API. There's one thing that might help here though, and that's increasing the NumMaxRetries at:

https://github.com/weaveworks/eksctl/blob/9bcfafe55ece4334a8cee10c6c27749959c1b021/pkg/eks/api.go#L134-L140

but since I can't reproduce this, I have no way to confirm this might help. If you or anyone else is still having this problem, let us know here and feel free to try increasing the NumMaxRetries, rebuilding eksctl and trying it again.

Closing for now.

golgoth31 commented 4 years ago

Hi, the problem has gone but I can't say when it has been resolved. Thank you

tarach commented 3 years ago

Hi, I've just encountered this error.

Since I'm learning I didn't bother at first to create an IAM account and I was using a root with eksctl and one issue aside it was working fine. ( I was able to create a cluster )

I've encountered an RBAC issue which I was hoping an account other than root will help me solve. I've created an account logged in to it via AWS Console and using aws configure command I've set access key and secret in .aws/credentials and after issuing command: eksctl create cluster --fargate --nodes-max 2 --name tutorial --region eu-central-1 --zones=eu-central-1a,eu-central-1b --verbose 5 which I've used previously and it was working fine I got the same error as described in this thread: "checking AWS STS access – cannot get role ARN for current session ..."

Using method described by @toahae above helped me solve this issue. ( ie. setting credentials via environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY )

gkhnsmsk commented 3 years ago

I found a workaround, I have to double quote the first ":" of the arn .... I can't understand why in my .aws/config file, my profile had a line like this: role_arn = arn:aws:iam::.... I change for that: role_arn = arn":"aws:iam::...... and the eksctl is ok

Why do you have role here? Cant you create a cluster with an admin user, or a user which has admin rights?

golgoth31 commented 3 years ago

I found a workaround, I have to double quote the first ":" of the arn .... I can't understand why in my .aws/config file, my profile had a line like this: role_arn = arn:aws:iam::.... I change for that: role_arn = arn":"aws:iam::...... and the eksctl is ok

Why do you have role here? Cant you create a cluster with an admin user, or a user which has admin rights?

This field allows me to assume role from one account to an other.

watcher00090 commented 3 years ago

I had this error also. It turned out that I had a typo in the cluster's region: I was putting an extraneous dash in front: I was writing -us-east-2 instead of us-east-2. Removing the extraneous dash made the error go away.

pabloa-aluna commented 3 years ago

Confirmed what @watcher00090 found. My region was wrongly typed as -r us-west2

I fixed to -r us-west-2 and it worked.

The error message is confusing.

sure-ally commented 3 years ago

I faced same issue but it's worked when I changed --region=${AWS_REGION} to --region==us-east-2. Somehow its not picking up variable ${AWS_REGION}.

NathanielRN commented 3 years ago

Can confirm that my --region us-west-2 was the problem. Not sure why it was, but removing that made the command work.

erjan commented 2 years ago

i tried these --region=us-east-1 --region==us-east-1

i added to ./aws/configure the line like

role_arn=aws":"iam.....

still no result! same error!