AttributeError: 'NoneType' object has no attribute 'arn'

jeremyjturner commented 4 years ago

I've reviewed #27 but it appears that I'm having the same issue.

Here are my steps to reproduce.

First, I'm using Docker version 19.03.8 on macOS:

jeremyturner: docker --version
Docker version 19.03.8, build afacb8b

I started the following container:

docker run -it --entrypoint /bin/ash hashicorp/terraform:latest

Changed to the home folder:

/ # cd ~/
~ # pwd
/root

Installed pip3:

echo "**** install Python ****" && \
    apk add --no-cache python3 && \
    if [ ! -e /usr/bin/python ]; then ln -sf python3 /usr/bin/python ; fi && \
    \
    echo "**** install pip ****" && \
    python3 -m ensurepip && \
    rm -r /usr/lib/python*/ensurepip && \
    pip3 install --no-cache --upgrade pip setuptools wheel && \
    if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi

Installed lambdaguard:

~ # pip3 install lambdaguard

~ # lambdaguard -V
2.4.1

In my case, I'm using JumpCloud as the IdP to my AWS account so I'm using a tool called SAML2AWS:

CURRENT_VERSION=2.25.0
wget https://github.com/Versent/saml2aws/releases/download/v${CURRENT_VERSION}/saml2aws_${CURRENT_VERSION}_linux_amd64.tar.gz
tar -xzvf saml2aws_${CURRENT_VERSION}_linux_amd64.tar.gz -C /bin/
chmod u+x /bin/saml2aws

Here is what the configuration looks like (small typo with the profile name):

~ # saml2aws configure -a LambdaGuardProfile
? Please choose a provider: JumpCloud
? AWS Profile LamdaGuardProfile
? URL https://sso.jumpcloud.com/saml2/aws-test-admin
? Username jeremyjturner@example.com

account {
  URL: https://sso.jumpcloud.com/saml2/aws-test-admin
  Username: jeremyjturner@example.com
  Provider: JumpCloud
  MFA: Auto
  SkipVerify: false
  AmazonWebservicesURN: urn:amazon:webservices
  SessionDuration: 3600
  Profile: LamdaGuardProfile
  RoleARN: 
}

Configuration saved for IDP account: LambdaGuardProfile

Now I login to the IdP to configure my .aws/credentials file:

~ # saml2aws login -a LambdaGuardProfile
Using IDP Account LambdaGuardProfile to access JumpCloud https://sso.jumpcloud.com/saml2/aws-test-admin
To use saved password just hit enter.
? Username jeremyjturner@example.com
? Password ************

Authenticating as jeremyjturner@example.com ...
? MFA Token 000000
Selected role: arn:aws:iam::XXXXXXXXXXXX:role/Admin
Requesting AWS credentials using SAML assertion
Logged in as: arn:aws:sts::XXXXXXXXXXXX:assumed-role/Admin/jeremyjturner@example.com

Your new access key pair has been stored in the AWS configuration
Note that it will expire at 2020-04-09 15:10:47 +0000 UTC
To use this credential, call the AWS CLI with the --profile option (e.g. aws --profile LamdaGuardProfile ec2 describe-instances).

Here we can confirm that the credentials are stored:

~ # cat .aws/credentials 
[LamdaGuardProfile]
aws_access_key_id        = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key    = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token        = BLAHBLAHETCETC
aws_security_token       = BLAHBLAHETCETC
x_principal_arn          = arn:aws:sts::XXXXXXXXXXXX:assumed-role/Admin/jeremyjturner@example.com
x_security_token_expires = 2020-04-09T15:10:47Z

When I run lambdaguard it seems to work:

~ # lambdaguard -v -p LamdaGuardProfile

         `.::////::.`
      ./osssssoossssso/.
    -osss/-`      .-/ssso-
  `osso-  .++++:      -osso`
 `oss/    .//oss-       /sss`
 +ss+        -sss.       /sso
.sss`       .sssso`      `sss.   LambdaGuard v2.4.1
-sso       :ssooss+       oss-
.sss`     /ss+``oss/     `sss.
 +ss+   `oss/   .sss///  /sso
 `oss/`.oso-     -ssso+./sso`
  `+sso:          .`  -oss+`
    -osss+-.`    `.-+ssso-
      ./osssssssssssso/.
         `.-:////:-.`

Loading regions (ap-east-1)
Loading regions (ap-northeast-1)
Loading regions (ap-northeast-2)
Loading regions (ap-south-1)
Loading regions (ap-southeast-1)
Loading regions (ap-southeast-2)
<snip>
Loading identity
          UserId......... AKIAIOSFODNN7EXAMPLE:jeremyjturner@example.com
          Account........ XXXXXXXXXXX
          Arn............ arn:aws:sts::XXXXXXXXXXX:assumed-role/Admin/jeremyjturner@example.com
[ 1/20 ] somethingsomethingFunction01
[ 1/20 ] somethingsomethingFunction02
         <snip>
         <snip>
          Security....... 0
          Triggers....... 0
          Resources...... 0
          Layers......... 0
          Runtimes....... 0
          Regions........ 0

          Report......... lambdaguard_output/report.html
          Log............ lambdaguard_output/lambdaguard.log

However, when I view the lambdaguard.log I get:

[2020-04-09 14:12] [arn:aws:lambda:ap-northeast-1:XXXXXXXXXXX:function:somethingsomethingFunction01]
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/lambdaguard/core/Lambda.py", line 63, in get_function
    if self.identity.acl.allowed("lambda:GetFunction"):
  File "/usr/lib/python3.8/site-packages/lambdaguard/utils/acl.py", line 97, in allowed
    simulation_results = self.client.simulate_custom_policy(
  File "/usr/lib/python3.8/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/client.py", line 612, in _make_api_call
    http, parsed_response = self._make_request(
  File "/usr/lib/python3.8/site-packages/botocore/client.py", line 632, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/lib/python3.8/site-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/lib/python3.8/site-packages/botocore/endpoint.py", line 132, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/usr/lib/python3.8/site-packages/botocore/endpoint.py", line 115, in create_request
    self._event_emitter.emit(event_name, request=request,
  File "/usr/lib/python3.8/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/signers.py", line 90, in handler
    return self.sign(operation_name, request)
  File "/usr/lib/python3.8/site-packages/botocore/signers.py", line 160, in sign
    auth.add_auth(request)
  File "/usr/lib/python3.8/site-packages/botocore/auth.py", line 357, in add_auth
    raise NoCredentialsError
botocore.exceptions.NoCredentialsError: Unable to locate credentials

[2020-04-09 14:12] [arn:aws:lambda:ap-northeast-1:XXXXXXXXXXXX:function:somethingsomethingFunction01]
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/lambdaguard/core/Lambda.py", line 192, in get_security
    self.report(),
  File "/usr/lib/python3.8/site-packages/lambdaguard/core/Lambda.py", line 216, in report
    'role': self.role.arn.full,
AttributeError: 'NoneType' object has no attribute 'arn'

So I tried again but this time by creating an AWS IAM user with an Access and Secret Key:

~ # lambdaguard -v -p LamdaGuardProfile

         `.::////::.`
      ./osssssoossssso/.
    -osss/-`      .-/ssso-
  `osso-  .++++:      -osso`
 `oss/    .//oss-       /sss`
 +ss+        -sss.       /sso
.sss`       .sssso`      `sss.   LambdaGuard v2.4.1
-sso       :ssooss+       oss-
.sss`     /ss+``oss/     `sss.
 +ss+   `oss/   .sss///  /sso
 `oss/`.oso-     -ssso+./sso`
  `+sso:          .`  -oss+`
    -osss+-.`    `.-+ssso-
      ./osssssssssssso/.
         `.-:////:-.`

Loading regions (ap-east-1)
Loading regions (ap-northeast-1)
Loading regions (ap-northeast-2)
<snip>
<snip>
Loading identity
          UserId......... AIDATQ2EXAMPLEBLAHETC
          Account........ XXXXXXXXXXXX
          Arn............ arn:aws:iam::XXXXXXXXXXXX:user/lambdaguard

[ 1/20 ] somethingsomethingFunction01
[ 1/20 ] somethingsomethingFunction02
<snip>
          Lambdas........ 0
          Security....... 0
          Triggers....... 0
          Resources...... 0
          Layers......... 0
          Runtimes....... 0
          Regions........ 0

          Report......... lambdaguard_output/report.html
          Log............ lambdaguard_output/lambdaguard.log

I'm getting the same error in the logs:

[2020-04-09 14:54] [arn:aws:lambda:ap-northeast-1:XXXXXXXXXXX:function:somethingsomethingFunction01]
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/lambdaguard/core/Lambda.py", line 63, in get_function
    if self.identity.acl.allowed("lambda:GetFunction"):
  File "/usr/lib/python3.8/site-packages/lambdaguard/utils/acl.py", line 97, in allowed  
    simulation_results = self.client.simulate_custom_policy(
  File "/usr/lib/python3.8/site-packages/botocore/client.py", line 316, in _api_call     
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/client.py", line 612, in _make_api_call
    http, parsed_response = self._make_request(
  File "/usr/lib/python3.8/site-packages/botocore/client.py", line 632, in _make_request 
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/lib/python3.8/site-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/lib/python3.8/site-packages/botocore/endpoint.py", line 132, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/usr/lib/python3.8/site-packages/botocore/endpoint.py", line 115, in create_request
    self._event_emitter.emit(event_name, request=request,
  File "/usr/lib/python3.8/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/lib/python3.8/site-packages/botocore/signers.py", line 90, in handler       
    return self.sign(operation_name, request)
  File "/usr/lib/python3.8/site-packages/botocore/signers.py", line 160, in sign
    auth.add_auth(request)
  File "/usr/lib/python3.8/site-packages/botocore/auth.py", line 357, in add_auth        
    raise NoCredentialsError
botocore.exceptions.NoCredentialsError: Unable to locate credentials

[2020-04-09 14:54] [arn:aws:lambda:ap-northeast-1:XXXXXXXXXXXX:function:somethingsomethingFunction01]
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/lambdaguard/core/Lambda.py", line 192, in get_security
    self.report(),
  File "/usr/lib/python3.8/site-packages/lambdaguard/core/Lambda.py", line 216, in report
    'role': self.role.arn.full,
AttributeError: 'NoneType' object has no attribute 'arn'

I thought maybe the problem was that I didn't have the AWS CLI installed so I tried that:

~ # pip install awscli
Collecting awscli
<snip>
~ # aws --version
aws-cli/1.18.39 Python/3.8.2 Linux/4.19.76-linuxkit botocore/1.15.39

However, the results are the same.

Perhaps I'm missing something simple?

Note that for the first assume role profile my IAM policy is full administrator and for the second IAM user with Access Key and Secret, the IAM policy was the AWS managed ReadOnlyAccess IAM policy.

antonbabenko commented 4 years ago

I have a similar error message when I use the role assumed with MFA.

Also, I got Missing both lambda:GetFunction and lambda:GetFunctionConfiguration. Not sure if it even the same problem or not.

adeptex commented 4 years ago

Hey @jeremyjturner, thanks a lot for this detailed report! I am not sure at all why this is happening. Some investigation effort is needed to get to the bottom of this. I've put it in our backlog and will try to include a fix in the next release.

adeptex commented 4 years ago

Hey @antonbabenko, thanks for confirming the issue.

You are getting that message because AWS keys that LambdaGuard uses do not have either permission allowed. So it can't get function info.

antonbabenko commented 4 years ago

I verified that I am having those permissions and retries couple of times. There should be something else.

bf-singularity commented 4 years ago

Hey @antonbabenko and @adeptex .

I am also getting the Missing both lambda:GetFunction and lambda:GetFunctionConfiguration error message. I did some troubleshooting and ran into some really, really weird results.

First off, the profile that I am running lambdaguard under definitely has the correct permissions - I verified through the AWS console and I am able to run both commands successfully from the AWS cli. By inserting various print() statements in the source code, I noticed that the list_policies_granting_service_access() call from the get_user_permissions() function in lambdaguard/utils/acl.py does not run on the profile that I specified through the --profile flag. Instead it runs against my default profile. Therefore, the call returns "error: An error occurred (NoSuchEntity) when calling the ListPoliciesGrantingServiceAccess operation: ARN arn:aws:iam::user/USER does not exist.", (note that I substituted USER for the actual user info). I verified that it runs on my default profile by inserting another self.client call right before it, and the results I get back are from the AWS environment configured under the default profile. So it's no surprise that that user isn't found, and therefore that the permissions aren't found.

The really weird thing is that I have no idea why this is happening. Again, by inserting print statements in various places I consistently see that the program is accepting the correct profile, is using the correct keys, has the correct ARN stored in the correct variables.... Everything looks perfectly fine right until those self.client calls are made, and then all of a sudden it's not running against the right profile.

I verified this problem on two separate virtual machines. Then I had a co-worker run the program for me instead, and it worked for him right out of the box without issues.

Please let me know what output I can provide you to help troubleshoot this problem.

Note that it's possible to bypass this issue by saving the access keys to the default AWS profile.

dquitmann-op commented 4 years ago

I can confirm:

FIX: saving the access keys in the default AWS profile bypasses this issue
changing https://github.com/Skyscanner/LambdaGuard/blob/ed323c9bc00b1941e0743f2d450a6ac85124fdb4/lambdaguard/core/STS.py#L26 to self.acl = ACL(self.caller['Arn'], profile, access_key_id, secret_access_key) fixed the issue for me. I think, the issue is caused by missing propagation of profile and keys at this point. But to be honest: I'm neither deep enough in the code nor in the AWS CLI at all to understand whats going on (eg. why is self.caller['Arn'] passed in this line instead of arn).

Hope this helps to debug the issue!

Thanks so much for the awesome tool otherwise, @adeptex

nwestfall commented 4 years ago

I had the same issue with passing --keys. I was able to reproduce in a fork of the project and found the issue. Thanks to @dhaug-op , the ACL class was the only class that needed these parameters for the base AWS class.

SergGu commented 4 years ago

We got the same Missing both lambda:GetFunction and lambda:GetFunctionConfiguration error with our first attempt to use LambdaGuard. We tried all kinds of authentication profiles and tricks from above post and then gave up.

adeptex commented 4 years ago

Hi all, the changes proposed by @nwestfall have been added. Please reset and reopen if the issue persists.

Thanks!

Skyscanner / LambdaGuard

AttributeError: 'NoneType' object has no attribute 'arn' #29