This PR fixes a memory usage issue whereby quota checks make a single boto3 call, even when supposedly paginating. Symptoms described in this comment on #31.
Uses the existing paginator to count the ebs_snapshot_count quota
Sets a default pagination page size (otherwise the paginator just gets all resources in my testing of describe_snapshots)
Fixes up an error log message
See memory usage spikes to ~800 MiB on 1.10.0 and is fairly stable at ~50 MiB on this branch.
Before:
Locally
$ time aws-quota-checker check ebs_snapshot_count
AWS profile: default | AWS region: ap-southeast-2 | Active checks: ebs_snapshot_count
EBS Snapshots per Region [xxx/ap-southeast-2]: 34896/100000 ✓
aws-quota-checker check ebs_snapshot_count 12.72s user 2.32s system 3% cpu 7:27.41 total
In cluster (~3 minutes):
AWS profile: default | AWS region: ap-southeast-2 | Active checks:
cf_stack_count,ebs_snapshot_count,rds_instances,s3_bucket_count
...
07-Jan-22 06:10:21 [INFO] aws_quota.prometheus - refreshing current values
07-Jan-22 06:13:39 [INFO] aws_quota.prometheus - current values refreshed
After:
Locally:
$ time aws-quota-checker check ebs_snapshot_count
AWS profile: default | AWS region: ap-southeast-2 | Active checks: ebs_snapshot_count
EBS Snapshots per Region [xxx/ap-southeast-2]: 34899/100000 ✓
aws-quota-checker check ebs_snapshot_count 12.33s user 1.36s system 16% cpu 1:25.01 total
In cluster (~1 minute):
AWS profile: default | AWS region: ap-southeast-2 | Active checks:
cf_stack_count,ebs_snapshot_count,rds_instances,s3_bucket_count
...
07-Jan-22 08:47:46 [INFO] aws_quota.prometheus - refreshing current values
07-Jan-22 08:48:55 [INFO] aws_quota.prometheus - current values refreshed
This PR fixes a memory usage issue whereby quota checks make a single
boto3
call, even when supposedly paginating. Symptoms described in this comment on #31.ebs_snapshot_count
quotadescribe_snapshots
)See memory usage spikes to ~800 MiB on![image](https://user-images.githubusercontent.com/13642138/148519980-41f9ec31-cae8-4147-bcf0-a5f6b4b64896.png)
1.10.0
and is fairly stable at ~50 MiB on this branch.Before:
Locally
After:
Locally: