API proposal: `tokio::task::try_spawn_blocking`

[lkts]

Is your feature request related to a problem? Please describe. Currently spawn_blocking queues tasks if there is no free threads to execute it in blocking thread pool. This behavior may not be suitable for some applications:

• Queuing is invisible to client code which can lead to surprising and hard to diagnose delays in blocking task execution. If there is a need to execute blocking tasks in latency-sensitive workflows, there needs to be a layer on top of Tokio to limit number of concurrent blocking tasks.
• Queue is not bounded which can in extreme cases lead to queue consuming all available memory.

Describe the solution you'd like I propose a try_spawn_blocking API.

pub fn try_spawn_blocking<F, R>(f: F) -> Result<JoinHandle<R>, NoAvailableThreads>
where
    F: FnOnce() -> R + Send + 'static,
    R: Send + 'static 
{
}

struct NoAvailableThreads {}

Alternatively (i don't know if it makes more sense implementation-wise but i suspect so) try_spawn_blocking can return a special BlockingTaskJoinHandle which will expose the error.

Describe alternatives you've considered Having a semaphore-based limiter on top of Tokio works well. Motivation for adding this API to Tokio is that:

• having such explicit API helps Tokio users to discover potential issues i am describing
• users don't have to handle (albeit not big) complexity of limiting concurrency on top of Tokio

[ipetkov]

Worth noting that with #4823 we have fallible spawn methods on task::Builder.

4850 is tracking its stabilization

[hawkw]

Yeah, since, as @ipetkov pointed out, the task::Builder API will have fallible spawn methods, I think the best way to add support for attempting to spawn a blocking task only if it wouldn't be queued is something like this:

task::Builder::new()
    .disable_blocking_queue() // name, obviously, subject to bikeshedding)
    .spawn_blocking(|| { /* ... */ }

and, we would add a QueueDisabled (again, name subject to bikeshedding) variant to SpawnError, here: https://github.com/tokio-rs/tokio/blob/cf181e8b1b29802cf9ef89d2777b653c04417351/tokio/src/runtime/blocking/pool.rs#L86-L92