nand2 / libvmod-throttle

Varnish: Reject or delay requests after given tresholds are reached. (Think API rate limit, or per-ip MISS rate limit)
Other
100 stars 22 forks source link

throttle sets no per ip limit, but global instead #14

Open intelbg opened 8 years ago

intelbg commented 8 years ago

Hello, I want to block xmlrpc and wp-login requests only for IP that floods and exceed the requests in secon, but when other people from other IPs opens the wp-login.php to be not affected as it's not. This is my configuration as it should be only per IP, but the limits actually are for all IPs that opens wp-login:

if(throttle.is_allowed("ip:" +req.http.X-Actual-IP, "45req/s") > 0s && (req.url ~ "xmlrpc.php|wp-login.php")) { error 429 "Calm down"; shield.conn_reset(); }

I tried also replacing req.httpd.X-Actual-IP with client.op, but then the limit does not work at all.

My question is why and how can I change it so the blocked IP to be only the ip that exceeds the requests, not all IP that opens wp-login.php. Thank you in advance.

nand2 commented 8 years ago

Hello!

Be careful, calling throttle.is_allowed() increments the counter, so here it increments for all URLs.

So first: if (req.url ~ "xmlrpc.php|wp-login.php") { if (throttle.is_allowed("ip:" +req.http.X-Actual-IP, "45req/s") > 0s) { shield.conn_reset(); } }

Then, the description of the bug makes me thing that "req.http.X-Actual-IP" is empty, so it is always "ip:", which makes the throttle global, whatever the IP. Can you check that req.http.X-Actual-IP works, by maybe printing it in the response headers?

Cheers, Nicolas

intelbg commented 8 years ago

So, if I understand my implementantion is worng and the right is the following, right?:

if (req.url ~ "xmlrpc.php|wp-login.php") { if (throttle.is_allowed("ip:" +req.http.X-Actual-IP, "45req/s") > 0s) { shield.conn_reset(); }

How can I print the respons header?

intelbg commented 8 years ago

Shouldn't it work with client.ip? When I set it to client.ip it doesn't work at all (but it's again global, not per ip ?).

nand2 commented 8 years ago

Hello!

Yes

if (req.url ~ "xmlrpc.php|wp-login.php") { if (throttle.is_allowed("ip:" +req.http.X-Actual-IP, "45req/s") > 0s) { shield.conn_reset(); }

Looks good.

Yes it should work with client.ip, as shown in the documentation... except if you use a CDN like Cloudflare. If you use a CDN, there should be a header given by your CDN with the real IP. Maybe it is your X-Actual-IP header you are using.

Anyhow, to help you debug, you can set headers in the response header, see for example in here https://www.varnish-cache.org/trac/wiki/VCLExampleHitMissHeader

sub vcl_deliver { set resp.http.X-Blah = req.http.X-Actual-IP; }

intelbg commented 8 years ago

From 1000 concurrent requests through apache benchmark to xmlrpc.php with the following configuration only 5 requests are failed from 1000 (and I always catch these requests that does not fails):

if ((req.url ~ "(wp-login.php|xmlrpc.php)")) { if(throttle.is_allowed("host:" + req.http.host, "10req/s") > 0s) { error 429 "Calm down"; shield.conn_reset(); } }

Is this a normal behavior and why so little number of requests is blocked? Is there an info that describe how these requests are handled and solution of this problem? If I set the throttle to 1 requests it works but this is super stupid and I can't understand why on 10 requests/s limit the throttle do nothing. If I can't solve this issue I should change the varnish with something other so I will be really thankfull if you help with this.