EKS Fargate profile usage check error

jwu2 commented 3 years ago

Bug Report

When reporting a bug in awslimitchecker, please provide all of the following information, as well as any additional details that may be useful in reproducing or fixing the issue:

Version

10.0.0

Installation Method

Docker

Supporting Software Versions

Docker

Actual Output

% docker run -e AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN jantman/awslimitchecker -S EKS -r ap-southeast-2 -u -vv
awslimitchecker 10.0.0 is AGPL-licensed free software; all users have a right to the full source code of this version. See <https://github.com/jantman/awslimitchecker>
2020-12-09 01:49:44,054 [DEBUG connectionpool.py:971 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): pypi.org:443
2020-12-09 01:49:44,116 [DEBUG connectionpool.py:452 - urllib3.connectionpool._make_request() ] https://pypi.org:443 "GET /pypi/awslimitchecker/json HTTP/1.1" 200 69866
2020-12-09 01:49:44,142 [DEBUG checker.py:285 - awslimitchecker.checker._boto_conn_kwargs() ] Connecting to region ap-southeast-2
2020-12-09 01:49:44,142 [DEBUG cloudtrail.py:125 - awslimitchecker.services.cloudtrail.get_limits() ] Gathering CloudTrail's limits from AWS
2020-12-09 01:49:44,204 [INFO connectable.py:119 - awslimitchecker.connectable.connect() ] Connected to dynamodb in region ap-southeast-2
2020-12-09 01:49:44,267 [INFO connectable.py:119 - awslimitchecker.connectable.connect() ] Connected to ec2 in region ap-southeast-2
2020-12-09 01:49:44,275 [INFO connectable.py:119 - awslimitchecker.connectable.connect() ] Connected to kinesis in region ap-southeast-2
2020-12-09 01:49:44,275 [DEBUG lambdafunc.py:84 - awslimitchecker.services.lambdafunc.get_limits() ] Getting limits for Lambda
2020-12-09 01:49:44,276 [DEBUG cloudtrail.py:125 - awslimitchecker.services.cloudtrail.get_limits() ] Gathering CloudTrail's limits from AWS
2020-12-09 01:49:44,276 [DEBUG lambdafunc.py:84 - awslimitchecker.services.lambdafunc.get_limits() ] Getting limits for Lambda
2020-12-09 01:49:44,284 [INFO connectable.py:119 - awslimitchecker.connectable.connect() ] Connected to support in region us-east-1
2020-12-09 01:49:44,284 [INFO trustedadvisor.py:147 - awslimitchecker.trustedadvisor.update_limits() ] Not using Trusted Advisor in regions outside of China or GovCloud; export FORCE_USE_TA=true to override.
2020-12-09 01:49:44,284 [DEBUG base.py:296 - awslimitchecker.services.base._update_service_quotas() ] Updating service quotas for EKS
2020-12-09 01:49:44,289 [INFO connectable.py:119 - awslimitchecker.connectable.connect() ] Connected to service-quotas in region ap-southeast-2
2020-12-09 01:49:44,289 [DEBUG quotas.py:79 - awslimitchecker.quotas.quotas_for_service() ] Getting service quotas for service code: eks
2020-12-09 01:49:44,311 [DEBUG connectionpool.py:971 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): servicequotas.ap-southeast-2.amazonaws.com:443
2020-12-09 01:49:44,423 [DEBUG connectionpool.py:452 - urllib3.connectionpool._make_request() ] https://servicequotas.ap-southeast-2.amazonaws.com:443 "POST / HTTP/1.1" 200 2408
2020-12-09 01:49:44,425 [DEBUG quotas.py:108 - awslimitchecker.quotas.quotas_for_service() ] Retrieved 8 quotas for service code eks: ['Clusters', 'Control plane security groups per cluster', 'Fargate profiles per cluster', 'Label pairs per Fargate profile selector', 'Managed node groups per cluster', 'Nodes per managed node group', 'Public endpoint access CIDR ranges per cluster', 'Selectors per Fargate profile']
2020-12-09 01:49:44,425 [DEBUG checker.py:432 - awslimitchecker.checker.find_usage() ] Finding usage for service: EKS
2020-12-09 01:49:44,425 [DEBUG eks.py:62 - awslimitchecker.services.eks.find_usage() ] Checking usage for service EKS
2020-12-09 01:49:44,432 [INFO connectable.py:119 - awslimitchecker.connectable.connect() ] Connected to eks in region ap-southeast-2
2020-12-09 01:49:44,434 [DEBUG connectionpool.py:971 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): eks.ap-southeast-2.amazonaws.com:443
2020-12-09 01:49:44,611 [DEBUG connectionpool.py:452 - urllib3.connectionpool._make_request() ] https://eks.ap-southeast-2.amazonaws.com:443 "GET /clusters HTTP/1.1" 200 66
2020-12-09 01:49:44,724 [DEBUG connectionpool.py:452 - urllib3.connectionpool._make_request() ] https://eks.ap-southeast-2.amazonaws.com:443 "GET /clusters/fargate_sydney HTTP/1.1" 200 2959
2020-12-09 01:49:44,818 [DEBUG connectionpool.py:452 - urllib3.connectionpool._make_request() ] https://eks.ap-southeast-2.amazonaws.com:443 "GET /clusters/fargate_sydney/node-groups HTTP/1.1" 200 34
2020-12-09 01:49:44,910 [DEBUG connectionpool.py:452 - urllib3.connectionpool._make_request() ] https://eks.ap-southeast-2.amazonaws.com:443 "GET /clusters/fargate_sydney/fargate-profiles HTTP/1.1" 200 112
2020-12-09 01:49:45,009 [DEBUG connectionpool.py:452 - urllib3.connectionpool._make_request() ] https://eks.ap-southeast-2.amazonaws.com:443 "GET /clusters/fargate_sydney/fargate-profiles/fargate_sydney-fargate-example HTTP/1.1" 200 688
Traceback (most recent call last):
  File "/usr/local/bin/awslimitchecker", line 33, in <module>
    sys.exit(load_entry_point('awslimitchecker', 'console_scripts', 'awslimitchecker')())
  File "/awslimitchecker/awslimitchecker/runner.py", line 550, in console_entry_point
    r.console_entry_point()
  File "/awslimitchecker/awslimitchecker/runner.py", line 492, in console_entry_point
    self.show_usage()
  File "/awslimitchecker/awslimitchecker/runner.py", line 318, in show_usage
    self.checker.find_usage(
  File "/awslimitchecker/awslimitchecker/checker.py", line 433, in find_usage
    cls.find_usage()
  File "/awslimitchecker/awslimitchecker/services/eks.py", line 66, in find_usage
    self._find_clusters_usage()
  File "/awslimitchecker/awslimitchecker/services/eks.py", line 142, in _find_clusters_usage
    label_pairs = selector['labels']
KeyError: 'labels'

In this case, the Fargate profile(s) doesn't have any labels in its selector, but has namespace specified.

Expected Output

Ideally returns without error, or at least catch and skip the error, so other checks can be continued.

Testing Assistance

Yes

sebasrp commented 3 years ago

I will look at this over the weekend at latest

sebasrp commented 3 years ago

Please test the commit below - I did check I was able to repro the issue through the unit test before applying the fix, but I do not have a production env like yours

jwu2 commented 3 years ago

I have tested with the same environment and can confirm the commit fixed the issue. Thank you @sebasrp!

jantman commented 3 years ago

A fix for this has been released in 11.0.0, which is now live on PyPI. Thank you so much, and apologies for the delay.

jantman / awslimitchecker