Account Assessment for AWS Organizations programmatically scans all AWS accounts in an AWS Organization for identity-based and resource-based policies with Organization-based conditions.
Apache License 2.0
25
stars
10
forks
source link
Queries to services do not return all resources because paging is not being used #8
Describe the bug
When working on accounts with large numbers of resources in a given service, only a portion of the resources are returned in the query. For example, if we have more than 100 IAM roles, the query will return only the first 100, as this is the default limit of the list_roles method page of the iam service in boto3. (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam/client/list_roles.html). This goes for other resources like IAM Policies, secrets in Secrets Manager, KMS Keys, SNS Topics, etc...
To Reproduce
Take an account over 100 IAM roles and start a Resource-Based Policies scan. You will only see the first 100 IAM roles.
Expected behavior
View all IAM roles in the account
Please complete the following information about the solution:
[v1.0.2] Version:
To get the version of the solution, you can look at the description of the created CloudFormation stack.
For example, "(SO0217) - The AWS CloudFormation hub template for deployment of the Account Assessment for AWS Organisations, Version: v1.0.0".
[us-east-1] Region: [e.g. us-east-1]
[no] Was the solution modified from the version published on this repository?
[n/a] If the answer to the previous question was yes, are the changes available on GitHub?
[n/a] Have you checked your service quotas for the sevices this solution uses?
[no] Were there any errors in the CloudWatch Logs?
Screenshots
If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).
Additional context
Add any other context about the problem here.
I collected the information I needed using a personal script and applied paging whenever the resource allowed. e.g:
#Get IAM roles
...
resource_list = []
paginator = client.get_paginator('list_roles')
page_iterator = paginator.paginate()
for page in page_iterator:
for item in page.get('Roles'):
if item is not None:
resource_list.append(item)
....
if resource_list is not None:
try:
for resource in resource_list:
....
Describe the bug When working on accounts with large numbers of resources in a given service, only a portion of the resources are returned in the query. For example, if we have more than 100 IAM roles, the query will return only the first 100, as this is the default limit of the list_roles method page of the iam service in boto3. (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam/client/list_roles.html). This goes for other resources like IAM Policies, secrets in Secrets Manager, KMS Keys, SNS Topics, etc...
To Reproduce Take an account over 100 IAM roles and start a Resource-Based Policies scan. You will only see the first 100 IAM roles.
Expected behavior View all IAM roles in the account
Please complete the following information about the solution:
To get the version of the solution, you can look at the description of the created CloudFormation stack.
For example, "(SO0217) - The AWS CloudFormation hub template for deployment of the Account Assessment for AWS Organisations, Version: v1.0.0".
Screenshots If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).
Additional context Add any other context about the problem here. I collected the information I needed using a personal script and applied paging whenever the resource allowed. e.g: