Optimal size of a thread pool

[suchenjm]

This project is very interesting. I wonder if the authors have considered how to measure the optimal size of a thread pool. As far as I know, there is currently no dataset available for this purpose.

[JohnStarich]

That's an interesting question, and if you find a compelling answer please consider sharing here!

This particular script defaults to a constant 50 threads and a CPU-core-count dependent number of processes at (2 * processor cores) + 1, though these are configurable.

My understanding is the optimal thread pool size highly depends on the kind of thread and the kind of work. From my experience with thread pools:

• For green (coroutine) threads, the number of threads is immaterial since they don't incur any system calls and there's minimal wait time on I/O before switching to a waiting thread. The only limit I might place here is on the memory required to hold everything in the waiting queue. For a 3GHz core, my rule of thumb is ~1000 green threads.
• For system threads and slow I/O-bound work, the optimal number of threads can be very high. As long as all threads are blocked, creating more threads can increase overall throughput.
• For system threads with fast I/O-bound work or CPU-bound work, the optimal number of threads should be a small multiple of the number of processor cores to prevent significant overhead of thread context-switching taking away precious clock cycles. My rule of thumb is 5-10 threads per processor core.