The maximum number of rules per security group has been reached

[lalau]

Looks like the cidr ranges have grown and once again reaches the limit of security group. This is what I get now running the example code:

Response:
{
  "errorMessage": "An error occurred (RulesPerSecurityGroupLimitExceeded) when calling the AuthorizeSecurityGroupIngress operation: The maximum number of rules per security group has been reached.",
  "errorType": "ClientError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 42, in lambda_handler\n    result = update_security_groups(ip_ranges)\n",
    "  File \"/var/task/lambda_function.py\", line 94, in update_security_groups\n    if update_security_group(client, group, new_ranges[\"GLOBAL\"], INGRESS_PORTS['Https']):\n",
    "  File \"/var/task/lambda_function.py\", line 136, in update_security_group\n    added += add_permissions(client, group, permission, to_add)\n",
    "  File \"/var/task/lambda_function.py\", line 170, in add_permissions\n    client.authorize_security_group_ingress(GroupId=group['GroupId'], IpPermissions=[add_params])\n",
    "  File \"/var/runtime/botocore/client.py\", line 316, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/runtime/botocore/client.py\", line 626, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
  ]
}

Request ID:
"4d2b63d4-4f60-4ea7-93aa-bf835a7ef1d4"

Function Logs:
nge: 52.66.194.128/26
Found CLOUDFRONT region: ap-southeast-1 range: 13.228.69.0/24
Found CLOUDFRONT region: us-east-2 range: 18.216.170.128/25
Found CLOUDFRONT region: us-east-1 range: 3.231.2.0/25
Found CLOUDFRONT region: ap-southeast-1 range: 52.220.191.0/26
Found CLOUDFRONT region: us-east-1 range: 34.232.163.208/29
Found CLOUDFRONT region: us-west-2 range: 35.162.63.192/26
Found CLOUDFRONT region: us-west-2 range: 34.223.80.192/26
Found CLOUDFRONT region: us-east-1 range: 34.226.14.0/24
Found CLOUDFRONT region: ap-northeast-1 range: 13.113.203.0/24
Found CLOUDFRONT region: ca-central-1 range: 99.79.168.0/23
Found CLOUDFRONT region: us-east-1 range: 34.195.252.0/24
Found CLOUDFRONT region: us-west-1 range: 52.52.191.128/26
Found CLOUDFRONT region: eu-west-2 range: 52.56.127.0/25
Found CLOUDFRONT region: us-west-2 range: 34.216.51.0/25
Found CLOUDFRONT region: ap-northeast-1 range: 52.199.127.192/26
Found CLOUDFRONT region: eu-west-1 range: 52.212.248.0/26
Found CLOUDFRONT region: ap-southeast-2 range: 13.210.67.128/26
Found CLOUDFRONT region: eu-central-1 range: 35.158.136.0/24
Found CLOUDFRONT region: eu-central-1 range: 52.57.254.0/24
Found CLOUDFRONT region: ap-northeast-2 range: 52.78.247.128/26
Found CLOUDFRONT region: eu-west-3 range: 52.47.139.0/24
Found 0 CloudFront_g HttpSecurityGroups to update
Found 1 CloudFront_g HttpsSecurityGroups to update
Found 0 CloudFront_r HttpSecurityGroups to update
Found 1 CloudFront_r HttpsSecurityGroups to update
sg-08c92bbebac0b0caf: Adding 120.52.22.96/27:443
sg-08c92bbebac0b0caf: Adding 180.163.57.128/26:443
sg-08c92bbebac0b0caf: Adding 120.253.240.192/26:443
sg-08c92bbebac0b0caf: Adding 116.129.226.128/26:443
sg-08c92bbebac0b0caf: Adding 223.71.71.128/25:443
sg-08c92bbebac0b0caf: Adding 120.253.245.128/26:443
sg-08c92bbebac0b0caf: Adding 210.51.40.0/24:443
sg-08c92bbebac0b0caf: Adding 58.254.138.0/25:443
sg-08c92bbebac0b0caf: Adding 116.129.226.0/25:443
sg-08c92bbebac0b0caf: Adding 120.52.39.128/27:443
sg-08c92bbebac0b0caf: Adding 118.193.97.64/26:443
sg-08c92bbebac0b0caf: Adding 223.71.71.96/27:443
sg-08c92bbebac0b0caf: Adding 180.163.57.0/25:443
sg-08c92bbebac0b0caf: Adding 223.71.11.0/27:443
sg-08c92bbebac0b0caf: Adding 36.103.232.128/26:443
sg-08c92bbebac0b0caf: Adding 111.51.66.0/24:443
sg-08c92bbebac0b0caf: Adding 120.52.153.192/26:443
sg-08c92bbebac0b0caf: Adding 119.147.182.0/25:443
sg-08c92bbebac0b0caf: Adding 120.232.236.0/25:443
sg-08c92bbebac0b0caf: Adding 58.254.138.128/26:443
sg-08c92bbebac0b0caf: Adding 120.253.245.192/27:443
sg-08c92bbebac0b0caf: Adding 120.52.12.64/26:443
sg-08c92bbebac0b0caf: Adding 36.103.232.0/25:443
sg-08c92bbebac0b0caf: Adding 119.147.182.128/26:443
sg-08c92bbebac0b0caf: Adding 118.193.97.128/25:443
sg-08c92bbebac0b0caf: Adding 120.232.236.128/26:443
sg-08c92bbebac0b0caf: Adding 120.253.241.160/27:443
[ERROR] ClientError: An error occurred (RulesPerSecurityGroupLimitExceeded) when calling the AuthorizeSecurityGroupIngress operation: The maximum number of rules per security group has been reached.
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 42, in lambda_handler
    result = update_security_groups(ip_ranges)
  File "/var/task/lambda_function.py", line 94, in update_security_groups
    if update_security_group(client, group, new_ranges["GLOBAL"], INGRESS_PORTS['Https']):
  File "/var/task/lambda_function.py", line 136, in update_security_group
    added += add_permissions(client, group, permission, to_add)
  File "/var/task/lambda_function.py", line 170, in add_permissions
    client.authorize_security_group_ingress(GroupId=group['GroupId'], IpPermissions=[add_params])
  File "/var/runtime/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/runtime/botocore/client.py", line 626, in _make_api_call
    raise error_class(parsed_response, operation_name)
END RequestId: 4d2b63d4-4f60-4ea7-93aa-bf835a7ef1d4
REPORT RequestId: 4d2b63d4-4f60-4ea7-93aa-bf835a7ef1d4  Duration: 909.64 ms Billed Duration: 1000 ms    Memory Size: 128 MB Max Memory Used: 83 MB  

[daniellwm]

Hi dear, simple request a limit raise at AWS, I just hit the same issue and now changed to 6 x 160 (6 SG per initerface, 160 rules per SG) than script ran normally

Cheers, Daniel

[raivirtual]

I solved this adding a new tag "EvenOrOdd" allowing to double the number of security groups.

But nevermind: I got CloudFront very slow after pointing ELB to the security groups with the IPs.

Perhaps is better to allow 0.0.0.0/0 and deny access by checking the header (adding a custom header with a key on CloudFront requests).

Anyway I will try to create a pull request here with the updates.

[raivirtual]

I realized that I was tired and that's why my CloudFront got "very slow" yesterday. I was associating my ELB SG to the 4 CF SG expecting for inheritance. Of course my CF wasn't able to even connect to the origin actually. It was not slow, it was timeout.

Now my ELB is directly associated to the 4 CF groups (with even and odd) and Lambda is updating the IPs automatically after my changes on the script. It's only 4 groups because I using only https protocol.

I sent a pull-request with the improvement. Hope it helps anybody else.

[imjeffparedes]

Now I'm stuck with this.

[vivekj11]

I solved this adding a new tag "EvenOrOdd" allowing to double the number of security groups.

But nevermind: I got CloudFront very slow after pointing ELB to the security groups with the IPs.

Perhaps is better to allow 0.0.0.0/0 and deny access by checking the header (adding a custom header with a key on CloudFront requests).

Anyway I will try to create a pull request here with the updates.

Hey @raivirtual Can you please help me with the changes that you made in code to add this new tag? Would be really helpful if you can share the changed snippet.

[wangerzi]

I got this problem too.....

[wangerzi]

Hi, @vivekj11 I solved this problem by create multi security group to save the ip rules, if your security group is not enougth, alert " Groups is not enough, we need x at least!".

You can see the latest code at https://github.com/wangerzi/aws-cloudfront-samples/blob/master/update_security_groups_lambda/update_security_groups.py

execute results:

[dchristian3188]

Another solution is to do this with AWS WAF and IPSets to get around the security group limits. We created this project that does just that.

https://github.com/aws-samples/aws-cloudfront-waf-ip-set

[vivekj11]

@wangerzi Thank you, I will definitely give it a try. In the meantime, I requested AWS to increase my SG rules limit and also created two separate SGs specifically for CloudFront, one for HTTP and other for HTTPS rules., So the limit is not a concern for now.

[karlskidmore]

A different and possibly simpler approach (create new SGs on each update and attach to ENIs then throw old away SGs), see here https://github.com/karlskidmore/SAM-AWS-IPs-to-SGs-to-ENIs -- been working in production for a while now with no hiccups. Hope this helps someone.