Does update_waf_ip_set() in the rate limiting Lambda always remove the currently blocked IPs even when the time limit has not passed?

[fieldju]

def update_waf_ip_set(outstanding_requesters, ip_set_id, ip_set_already_blocked):
print "[update_waf_ip_set] Start"

counter = 0
try:
    if ip_set_id == None:
        print "[update_waf_ip_set] Ignore process when ip_set_id is None"
        return

    updates_list = []
    waf = boto3.client('waf')

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tTruncate [if necessary] list to respect WAF limit"
    #--------------------------------------------------------------------------------------------------------------
    top_outstanding_requesters = {}
    for key, value in sorted(outstanding_requesters.items(), key=lambda kv: kv[1]['max_req_per_min'], reverse=True):
        if counter < LIMIT_IP_ADDRESS_RANGES_PER_IP_MATCH_CONDITION:
            if not is_already_blocked(key, ip_set_already_blocked):
                top_outstanding_requesters[key] = value
                counter += 1
        else:
            break

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tRemove IPs that are not in current outstanding requesters list"
    #--------------------------------------------------------------------------------------------------------------
    response = waf_get_ip_set(ip_set_id)
    if response != None:
        for k in response['IPSet']['IPSetDescriptors']:
            ip_value = k['Value'].split('/')[0]
            if ip_value not in top_outstanding_requesters.keys():
                updates_list.append({
                    'Action': 'DELETE',
                    'IPSetDescriptor': {
                        'Type': 'IPV4',
                        'Value': k['Value']
                    }
                })
            else:
                # Dont block an already blocked IP
                top_outstanding_requesters.pop(ip_value, None)

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tBlock remaining outstanding requesters"
    #--------------------------------------------------------------------------------------------------------------
    for k in top_outstanding_requesters.keys():
        updates_list.append({
            'Action': 'INSERT',
            'IPSetDescriptor': {
                'Type': 'IPV4',
                'Value': "%s/32"%k
            }
        })

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tCommit changes in WAF IP set"
    #--------------------------------------------------------------------------------------------------------------
    response = waf_update_ip_set(ip_set_id, updates_list)

except Exception, e:
    print "[update_waf_ip_set] Error to update waf ip set"
    print e

print "[update_waf_ip_set] End"
return counter

This code appears to take the list of IPS that we determined that we needed to block / keep blocked and make a new list of IPs that are not already blocked.

               if not is_already_blocked(key, ip_set_already_blocked):
                top_outstanding_requesters[key] = value
                counter += 1

Using that list it appears as if we delete from the ip set anything that is not in this new list, which would include any ips that should still be blocked.

                if ip_value not in top_outstanding_requesters.keys():
                updates_list.append({
                    'Action': 'DELETE',
                    'IPSetDescriptor': {
                        'Type': 'IPV4',
                        'Value': k['Value']
                    }
                })

Wouldn't everything in the top list not be in the ip set and since we only add to it if its not already blocked. We then immediately remove everything thats not in the top list from the ip set so we are really clearing the ip set every time.

Am I reading this right?

[fieldju]

I'm 90% sure, I read things wrong and it's just excluding IPs that are in the manual IP set.

I will close this.