Add anti DoS heuristics

[ilya-stromberg]

Edit: Original title: Specify how to control amount of threads and fibers

@s-ludwig, is it possible to control amount of threads and fibers? If yes, please specify how to do it. It's important because: 1) we must control processor resources. 2) program can support only 1 thread, so multithreaded mode is not supported.

[s-ludwig]

Currently it's a boolean choice - by default only one thread with an infinite number of fibers is used. If enableWorkerThreads is called (this function will be removed eventually) then using runWorkerTask instead of runTask will execute a task in one of the worker threads. The number of worker threads always equals the number of CPU cores.

[ilya-stromberg]

OK, thanks. Please document it. I didn't find any remarks at the http://vibed.org/docs page.

Additional question: what about multithreaded mode support for Vibe.d plugin? For example, I have global variable. Must I use synchronization (Mutex and/or atomic operations) if I have only one thread (default in Vibe.d)?

And the last question: does fiber supports thread-local variable (1 thread-local variable for each fiber), or it's still per-thread variable (only 1 variable because Vibe.d has only one thread by default)?

[extrawurst]

see the get/set of the task class here: http://vibed.org/api/vibe.core.task/TaskFiber

On Mon, Nov 25, 2013 at 4:55 PM, ilya-stromberg notifications@github.comwrote:

OK, thanks. Please document it. I didn't find any remarks at the http://vibed.org/docs page.

Additional question: what about multithreaded mode support for Vibe.d plugin? For example, I have global variable. Must I use synchronization (Mutex and/or atomic operations) if I have only one thread (default in Vibe.d)?

And the last question: does fiber supports thread-local variable (1 thread-local variable for each fiber), or it's still per-thread variable (only 1 variable because Vibe.d has only one thread by default)?

— Reply to this email directly or view it on GitHubhttps://github.com/rejectedsoftware/vibe.d/issues/405#issuecomment-29212883 .

[s-ludwig]

Since 0.7.17 there is also TaskLocal!T, which is recommended now (the old Variant based get/set methods will get deprecated eventually as they are slower and don't support static type checking).

For default thread local variables you don't have to used any synchronization - they can be safely shared between tasks that run in the same thread (all tasks started with runTask). However, if this variable is a complex type that does any kind of I/O operations or calls yield(), it needs to be protected by a vibe.core.sync.TaskMutex or similar.

I'll keep the ticket open to remember extending the docs.

[ilya-stromberg]

However, if this variable is a complex type that does any kind of I/O operations or calls yield()

Do you mean Vibe.d asynchronous I/O? Do I still need to use Mutex if I use classic blocking I/O?

[s-ludwig]

Async I/O (i.e. anything that yields execution or needs the event loop) - Basically TaskMutex works exactly like Mutex, but does not block the event loop when it needs to wait. For classic blocking, unbuffered I/O (which shouldn't be mixed with async. I/O, though) no mutex is necessary as it is synchronized internally.

[ilya-stromberg]

For default thread local variables you don't have to used any synchronization - they can be safely shared between tasks that run in the same thread

Can fiber migrate from one thread to another (for multithreaded mode if enableWorkerThreads is called)?

[s-ludwig]

No, they always stay on the thread where they have been started. They have to, so that they don't violate D's threading model (i.e. un-shared references would be shared across threads otherwise).

[ilya-stromberg]

Currently it's a boolean choice - by default only one thread with an infinite number of fibers is used.

We must provide an ability to limit number of fibers. If we have too many started fibers it means that server is overloaded. It's better to send 503 error ("I'm busy right now, sorry."), than delay all answers.

See also: https://hacks.mozilla.org/2013/01/building-a-node-js-server-that-wont-melt-a-node-js-holiday-season-part-5/

It's Node.js plugin description that illustrates the same idea, but they use time-based limits. Probably, it's better, but requests more resources. We can implement both.

[mihails-strasuns]

Does not really sound convincing. This is job of load balancer, not server itself.

[s-ludwig]

The load factor doesn't necessarily depend on the number of fibers itself, primarily it can become an issue for memory usage. Maybe a better heuristic is the number of enqueued events in the event loop plus the number of manually yielded fibers. But ultimately it may be difficult to fine tune it, so average maximum request time or something similar may be more robust.

Another thing to remember is that responding to requests with 503 may require as much (or nearly as much) resources as the original request would have and thus may not avoid slowing down the server further. This would be an argument for @Dicebot's point of view. It would also be interesting to reject requests (or even connections if things come really bad) based on IP address to e.g. keep clients that have already been served successfully working and only reject new clients, but such a logic would definitely be too much for the core system.

Anyway, I think it is still important to add a few overload protections, primarily to help against certain kinds of (D)DoS attacks.

[s-ludwig]

The approach used by node-toobusy of measuring event loop latency also sounds interesting and would work exactly like that for vibe.d.

[ilya-stromberg]

This is job of load balancer, not server itself.

OK. Assume that all our servers busy (load balancer does great job). What should we do with overload? Also, note that the load balancer can select another server if first one sent an error. For example, nginx can do this for fastCGI/sCGI backends.

The load factor doesn't necessarily depend on the number of fibers itself, primarily it can become an issue for memory usage.

Yes, I agree. As I said, Mozilla use time-based limits. But for many applications you can calculate average time per request. So, we can use both time-based and count-based limits, but setup count-based limit 2-5 times bigger than the time-based limit. So, in normal application run we will detect overload via time-based limit. But time-based limit can't help in sharp load factor change (guys, (D)DoS is coming) because it calculated every 0.5-1 sec. So, count-based limit can help in this case, because hacker can't add more than X tasks.

Another thing to remember is that responding to requests with 503 may require as much (or nearly as much) resources as the original request would have and thus may not avoid slowing down the server further.

Please, read original article. The main idea to close connection as soon as possible without any database requests and so on. It helps (at least in that case).

[ilya-stromberg]

The approach used by node-toobusy of measuring event loop latency also sounds interesting and would work exactly like that for vibe.d.

Great! Thanks that you read it. @Dicebot, we have Russian translation, you can see it if you want: http://habrahabr.ru/company/nordavind/blog/196518/

[mihails-strasuns]

@ilya-stromberg Thanks but I do prefer English for tech stuff ;)

OK. Assume that all our servers busy (load balancer does great job). What should we do with overload?

Whatever is configured on load balancer. Try best effort, reject, raise an alarm - your choice. Problem with delegating this decision to server itself that it becomes harder to scale server count transparently. What is useful though is the built-in ability to report current load to the balancer.

[ilya-stromberg]

What is useful though is the built-in ability to report current load to the balancer.

OK. If I understand you correctly, we need that server 1) can detect current load factor and send this information to the load balancer (of course, if we have load balancer) 2) can detect his overload and reject request (of course, if we don't have load balancer because, for example, we have only one server)

Is it correct?

[mihails-strasuns]

We need a server being able to detect its load factor and let programmer decide what he wants to do here ;)

[ilya-stromberg]

Mozilla's article describes simple way to do this. Sorry if I confused public API and it's usage, but your first answer looks like server doesn't need to calculate load factor at all. I believe that now everything is clear.

[s-ludwig]

I'd like to make this a separate package ("vibe-overload"?) if possible. It may need some additional hooks, but it looks like it can be nicely self-contained. The main reason is that the rest of vibe.d should also get split up (core, http, mongodb, redis, mail, diet).

[ilya-stromberg]

OK, thanks. Do you want to implement only time-based limit or both time-based and count-based limits?

[s-ludwig]

I think nothing speaks against using multiple limits.

[mihails-strasuns]

Sorry if I confused public API and it's usage, but your first answer looks like server doesn't need to calculate load factor at all. I believe that now everything is clear.

Yes I was objecting only to hard-wiring limits to 50X page.

[ilya-stromberg]

If enableWorkerThreads is called (this function will be removed eventually)

Do you have any problems with it? Or you just want to change API?

[s-ludwig]

It will be enabled by default. It used to have side effects when it was an experimental feature, but nowadays it doesn't hurt as long as distributed TCP listening or runWorkerTask isn't explicitly used.

[Geod24]

One effort in that direction: https://github.com/vibe-d/vibe.d/pull/2496