L7 IP blocking

[krizhanovsky]

Currently filter module just filters IPs loaded by tfw_filter_block_ip() and this is duplication of nftables functionality. However, we need this to quickly load up to 1M IP filtering rules, managing them and provide persistency on reboot (i.e. if we dynamically load 1M rules and then reboot the server, all the rules must be reloaded on the server startup). nftables and IPset partially address these issues, so need to replace filter module by an interface to nftables and probably patch nftables to provide them quick index and persistency. The final filter data set must have quick index for the rules and be preferably NUMA-aware.

Action interface for different filtration strategies

Actually, nftables isn't the only and most efficient way to filter DDoS attacks. Instead of generating nftable rule. it has sense to generate/modify an XDP program. The other frequent way to defeat DDoS is to announce blocked IPs via BGP to a router. So the filter must implement a configuration option (the exact syntax is to be discussed):

filter <filter_name> <action>

, so HTTP tables plus block action (blocking traffic on application layer) can use filter_name (this must be implemented) to run custom action and block traffic on L3 and lower layers. <filter_name> is just a string identifier. <action> should define exact nftable, XDP or user space script call (this is also TBD).

[krizhanovsky]

Aggregate the issue with #1042 - filtering IP on XDP:

Recent research shows that filtering single IP address can be ten times more efficient with XDP than with iptables. That can be handy to mitigate DDoS attacks.

[i-rinat]

Currently (1617736e464eab3c1ffe8119b1b132ede519fd7c), tfw_filter_block_ip() uses simple hashing tfw_ipv6_hash():

static unsigned long
tfw_ipv6_hash(const struct in6_addr *addr)
{
        return ((unsigned long)addr->s6_addr32[0] << 32)
               ^ ((unsigned long)addr->s6_addr32[1] << 24)
               ^ ((unsigned long)addr->s6_addr32[2] << 8)
               ^ addr->s6_addr32[3];
}

which is fine for IPv4 addresses, but dangerous for IPv6. Clients with IPv6 usually have /64 (or in rare cases, /48) subnet at their disposal, which gives them control over at least 64 bits of the address.

Such clients can (1) circumvent address blocking, or (2) poison storage for IPs by overloading particular buckets. We need to prevent that.

One of the solutions is to always discard last 64 bits of IPv6 address. In other words, always block whole /64 subnets.

[krizhanovsky]

Introduction

The initial task statement is wrong for Tempesta design: we're L7 firewall and we should be able to filter IP addresses came through L7 proxies. So instead of current logic mimic L3 firewall we must use real client IP from X-Forwarded-For, X-Real-IP or Forwarded: for headers. So depends on #1350 (Forwarded HTTP header by RFC 7239).

We need to implement real IP definition taking into account trusted sources. Also an attacker could cause a confusion e.g. by setting all the headers at the same time to different values. Such and similar cases must handled.

The task is partially a bug since we can not correctly filter traffic if Tempesta FW there is a proxy between a client and Tempesta FW.

Both the initial (from skb) and resolved (from all the headers) client IPs must be written to access log.

New config options

All the variables must be global and should be reconfigurable on the fly.

We have #1350 parsing standard headers, so we don't need to support Nginx's real_ip_header option. Client IP resolving must be recursive (i.e. stop on first untrusted IP) and the search always must scan HTTP headers in the same order.

Need to implement a new trusted_ip_from config option mimicing http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from . Only IP (e.g. 1.1.1.1) or subnets (e.g. 1.1.1.0/24) in IPv4 and IPv6 must be supported.

Use 4-bit per level Linux radix tree or TDB HTtrie with a modification to keep 8 layers instead of current constant 16 (subnets just use zeroes on last levels).

The new filter logic

Now filter.c must use the resolved client IP for blocking. The filter.c and http_limit.c logic must be called after IP resolving, so if there is no trusted_ip_from, then the whole logic is just equivalent for the current one.

Documentation

The image https://github.com/tempesta-tech/tempesta/wiki/DDoS-mitigation obviously becomes false and must be updated.

The new config option must be described in https://github.com/tempesta-tech/tempesta/wiki/Handling-clients

If reconfiguration on the fly is implemented, then https://github.com/tempesta-tech/tempesta/wiki/On-the-fly-Reconfiguration must also be updated.

Testing

Test for the issue https://github.com/tempesta-tech/tempesta-test/issues/216

[krizhanovsky]

2074 disables the client User-Agent logic and relies on IP addresses only, so multi-CDN and other proxying in front of Tempesta FW cause identification problems. So move to earlier milestone