Add priority to spawned tasks

[zamazan4ik]

Is your feature request related to a problem? Please describe. I have a large bunch of tasks. These tasks have different prioroties for me - some of them are more important for me, so I want to execute them as soon as possible. As a possible solution for the problem can be task priorities.

Describe the solution you'd like I am not really sure with desired API, but I think about something like this one: tokio::spawn(priority, task)

Describe alternatives you've considered Honestly, I see no alternatives here.

[Darksonn]

I recall reading that linkerd uses multiple runtimes to make sure that tasks on one cannot starve tasks on the other.

[zamazan4ik]

Really multiple runtimes cannot resolve such problem. If we create two runtimes: one for high-priority tasks with M workers and another one with K workers for low-priority tasks and M + K == CPU cores, we can guarantee, that two runtimes will not interleave. But in this case we in some cases will not utilize all CPU resources, if high or low priority runtimes are not loaded enough.

If we will create runtimes with workers where M + K > CPU cores, we cannot guarantee, that runtime with high-priority tasks will execute its before low-priority, if both runtimes have enough tasks to do.

But after your answer I just came up with another idea: set priority not for tasks but for runtimes :). However, in this case we possibly can get some overhead of having two runtimes in the app instead of one.

[averyanalex]

An example of creating runtimes with different priorities (perhaps it will be useful to someone):

fn spawn_runtime(priority: ThreadPriority, policy: ThreadSchedulePolicy) -> Runtime {
    let thread_id = thread_native_id();

    let current_priority = get_current_thread_priority().unwrap();
    let current_policy = thread_schedule_policy().unwrap();

    if set_thread_priority_and_policy(thread_id, priority, policy).is_err() {
        warn!("can't set policy {policy:?} with priority {priority:?}");
    };
    let runtime = tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .unwrap();
    info!("starter runtime with scheduling policy {policy:?} and priority {priority:?}");
    if set_thread_priority_and_policy(thread_id, current_priority, current_policy).is_err() {
        warn!("can't set policy {policy:?} with priority {priority:?}");
    };
    runtime
}

static HIGH_PRIORITY_RUNTUME: LazyLock<Runtime> = LazyLock::new(|| {
    spawn_runtime(
        22.try_into().unwrap(),
        ThreadSchedulePolicy::Realtime(RealtimeThreadSchedulePolicy::RoundRobin),
    )
});
static NORMAL_PRIORITY_RUNTUME: LazyLock<Runtime> = LazyLock::new(|| {
    spawn_runtime(
        21.try_into().unwrap(),
        ThreadSchedulePolicy::Realtime(RealtimeThreadSchedulePolicy::RoundRobin),
    )
});

[LeGamerDc]

look like priority will make task schedule cost more (rbtree instead of fifo). and most scene for task is very short cpu, and add priority sure will affect schedule throughput.

[Ten0]

rbtree instead of fifo

Looks like this would be binary heap, not rb tree. That's only a factor 2 of perf. between the two though.

[Darksonn]

Neither a binary heap or rb tree work well with work stealing, which is necessary for the multi threaded runtime.

Also, an absolute priority such as a binary heap or rb tree is unlikely to be what you want. It allows a high priority task to completely prevent a lower priority task from running, but that kind of starvation is pretty unlikely to be what you want.