[Feature] Async thread pool

[jeremyong]

For work that needs to perform I/O or block on an external activity, currently, the user is responsible for managing additional threads (possibly set to a lower priority) that perform work wrapped in an async_node. The general pattern for such work is:

1. Configurable, async thread-pool size that is typically greater than the hardware concurrency available
2. A "flex" parameter that allows the async thread pool to exceed its steady-state size to accommodate additional work if necessary
3. An API that transparently maps work to a leased async thread
4. Optionally pinning async threads to a particular core. For example, I/O threads typically should be isolated to prevent cache pollution since no reads will be hot by definition.
5. Configurable lower prioritization to accommodate the situations in which most instructions on such threads will be waiting or busy-waiting

I think such a feature would greatly streamline the use cases for async_node in a way that is scalable and broadly applicable. I would say the first 3 items in the list above are likely a requirement for an MVP, and the 4th and 5th items would be nice-to-haves.

I wouldn't mind having a go at this myself, but wanted to create the issue first to gauge interest, or to determine if this was already considered (or even in progress).

Thanks

[alexey-katranov]

The idea seems reasonable. Thank you.

[arunparkugan]

@alexey-katranov is this issue is still relevant?