search

crowdsecurity / cs-aws-waf-bouncer

Crowdsec bouncer for AWS WAF
MIT License
7 stars 5 forks source link

ThrottlingException few times a day #16

Open chladic opened 1 year ago

chladic commented 1 year ago

Hello crowdsec team,

few times a day I get error:

Failed to update IPSets: Failed to commit ipset changes: ThrottlingException: Rate exceededntstatus code: 400, request id: xxxxxxxxxxxxx" acl=my_waf_acl region=eu-central-1 scope=REGIONAL

AWS support told me there is hard limit to do 1 API call per second. Is it possible this is creating more calls per second ?

image

Thanks a lot

blotus commented 1 year ago

Hello,

This can happen if you have multiple elements in your waf_config or maybe during the initial creation of the sets at startup if you have more than 10k decisions (although i've never ran into this limit).

https://github.com/crowdsecurity/cs-aws-waf-bouncer/pull/15 should help reduce the amount of calls made to the UpdateIPSet API when sharing the same RuleGroup for multiple WebACLs.

We'll be looking into implementing rate limiting when calling AWS APIs to make sure we don't go over the quotas.

chladic commented 1 year ago

Hi @blotus. I have only one web_acl in my config

waf_config:
  - web_acl_name: web-waf
    fallback_action: captcha
    rule_group_name: crowdsec-rule
    scope: REGIONAL
    region: ...
    ipset_prefix: crowdsec-ipset
    capacity: 50

Regarding decisions I have only 50-100 (+ community blacklist where is like 26k IP addresses).

I use workaround what you suggested where in web-waf2 Im using crowdsec RuleGroup created in web-waf (manually added). This can cause that bouncer is using more API calls ?