Bouncer every 2 hours creates duplicated IPset in WAF for community blocklist

[chladic]

Hello crowdsec,

When I start bouncer it creates 2 IP sets for community blocklist (cca 18000 IPs, so 10000 + 8000), then it creates IPset for custom Ban ipv4 + custom ipset for ipv6 + custom ipset for captcha. In total 5 IPsets what is fine.

But then every 2 hours it creates another IPset with community blocklist. I found out that bouncer take alert whenever community blacklist is updated but instead of replacing it, it creates new one.

cscli alerts list|head -40|grep update
| 86 | crowdsecurity/community-blocklist | update : +14249/-0 IPs            |         |                                | ban:14249 | 2022-06-27 13:03:36 +0000 UTC  |
| 83 | crowdsecurity/community-blocklist | update : +14248/-0 IPs            |         |                                | ban:151   | 2022-06-27 11:03:36 +0000 UTC  |
| 75 | crowdsecurity/community-blocklist | update : +14220/-0 IPs            |         |                                | ban:126   | 2022-06-27 09:03:36 +0000 UTC  |

time="27-06-2022 09:04:13" level=info msg="Creating IPSet crowdsec-ipset-us-east-1-IPV4-ban-2c64b9a0-bae4-4402-98d7-b92f58db48d9" set=crowdsec-ipset-us-east-1-IPV4-ban-2c64b9a0-bae4-4402-98d7-b92f58db48d9
time="27-06-2022 11:04:04" level=info msg="Creating IPSet crowdsec-ipset-us-east-1-IPV4-ban-83c588a1-c110-4fa3-bc29-13e2b304a8b6" set=crowdsec-ipset-us-east-1-IPV4-ban-83c588a1-c110-4fa3-bc29-13e2b304a8b6
time="27-06-2022 13:04:03" level=info msg="Creating IPSet crowdsec-ipset-us-east-1-IPV4-ban-ade87bb1-e63a-485b-8707-4c0ff428bf89" set=crowdsec-ipset-us-east-1-IPV4-ban-ade87bb1-e63a-485b-8707-4c0ff428bf89

[blotus]

Hello,

This is definitely something that should not happen :/

Could you upload the logs from the bouncer somewhere to see if we got an error response from the WAF API at any point ?

[chladic]

@blotus

There is this msg as well:

time="27-06-2022 09:03:52" level=info msg="could not find empty set, creating new set" acl=waf-production-alb-web-waf component=ipset_manager region=eu-west-1 scope=REGIONAL
time="27-06-2022 09:04:12" level=info msg="could not find empty set, creating new set" acl=waf-production-alb-web-waf component=ipset_manager region=eu-west-1 scope=REGIONAL
time="27-06-2022 11:04:02" level=info msg="could not find empty set, creating new set" acl=waf-production-alb-web-waf component=ipset_manager region=eu-west-1 scope=REGIONAL
time="27-06-2022 13:04:02" level=info msg="could not find empty set, creating new set" acl=waf-production-alb-web-waf component=ipset_manager region=eu-west-1 scope=REGIONAL

[buixor]

cf. https://github.com/crowdsecurity/cs-aws-waf-bouncer/pull/10

[blotus]

Fixed by #10.

[chladic]

@blotus fantastic, many thanks