search

rust-lang / libs-team

The home of the library team
Apache License 2.0
116 stars 18 forks source link

OS-level `thread::Builder` priority and affinity extensions #195

Open ian-h-chamberlain opened 1 year ago

ian-h-chamberlain commented 1 year ago

Proposal

Supersedes #71. This proposal is based on the discussion there and on Zulip about that proposal.

cc @AzureMarker @joshtriplett as they were involved in the earlier discussion / proposal as well.

Problem statement

Exposing OS thread scheduling options has been requested for quite a long time. Some third-party crates have been created to supplement the std functionality, but they seem to mostly work by either duplicating std APIs or only work for the current thread after spawning.

This proposal aims to enable std::os extension traits that can modify thread::Builder to set some of these properties before a thread is spawned, without committing to a higher-level cross-platform API (but ideally leaving room for one to be designed and implemented in the future).

Motivation, use-cases

Setting thread affinity and priority has a variety of motivations. For platforms that use a cooperative thread scheduler, setting CPU affinity and priority may be necessary to ensure that threads are not starved or cause deadlocks. High-performance / realtime applications may need fine grained control over their threads to meet performance requirements.

In principle, I believe this proposal should enable implementation of the necessary APIs to exert low-level OS-specific control over threads, which should pave the way for a more general-purpose cross-platform solution for these use cases.

Solution sketches

I have taken some of the changes from previous attempts and implemented them in a fork here, trying to show the minimum viable API surface:

The new public API surface area is fairly small but can be expanded on a per-platform basis as support is added / desired:

std::os::horizon::thread

/// Horizon-specific extension trait for [`thread::Builder`](crate::thread::Builder).
pub trait BuilderExt {
    /// Set the priority of the thread to be spawned.
    ///
    /// See <https://www.3dbrew.org/wiki/Multi-threading#Threads> for details
    /// about the meaning / valid values of the `priority` parameter.
    fn priority(self, priority: libc::c_int) -> Self;
}

impl BuilderExt for crate::thread::Builder {}

std::os::linux::thread

/// Linux-specific extensions for [`thread::Builder`](crate::thread::Builder).
pub trait BuilderExt {
    /// Set the CPU affinity (which cores to run on) for the thread to be spawned.
    ///
    /// See <https://man7.org/linux/man-pages/man3/CPU_SET.3.html> for more details
    /// about how to construct the `cpu_set` parameter.
    fn affinity(self, cpu_set: libc::cpu_set_t) -> Self;
}

impl BuilderExt for crate::thread::Builder {}

Draft of an implementation here:

https://github.com/rust-lang/rust/compare/master...ian-h-chamberlain:rust:feature/thread-schedule-os-ext

Links and related work

the8472 commented 1 year ago

Why are those on the thread-builder though? Thread priorities and affinities can be set after creation on most platforms.

A more general and portable mechanism would be providing a closure that can wrap the Builder::spawn closure, similar to CommandExt::pre_exec (but safer).

And the mix of proposed horizon and linux APIs is weird.

ChrisDenton commented 1 year ago

See Processor Groups and Scheduling Priorities for Windows considerations.

Since (I assume) most libstd platforms support some form of setting priorities and/or affinities, surely there most be some hope of cross-platform APIs? Within the standard library I personally really prefer cross-platform APIs if at all possible because it provides the most value. Users get cross-platform support "for free" just by using normal APIs.

ian-h-chamberlain commented 1 year ago

Why are those on the thread-builder though? Thread priorities and affinities can be set after creation on most platforms.

That's true, this proposal is specifically limited to the Builder (setting these attributes before spawning) as I didn't want to overextend the scope. I could see potential for a similar extension trait on std::thread::Thread as well in the future.

A more general and portable mechanism would be providing a closure that can wrap the Builder::spawn closure, similar to CommandExt::pre_exec (but safer).

This is a fair point and something I hadn't considered. Even with this mechanism though, I think OS-specific helpers would need to be provided to expose the underlying structures that control the spawning of the thread. Maybe it would end up looking something like this?

use std::os::linux::thread::affinity;

let cpus: libc::cpu_set_t = todo!();

let t = std::thread::Builder::new()
    .with(affinity(cpus))
    .spawn(|| println!("hi"))
    .unwrap();

t.join().unwrap();

And the mix of proposed horizon and linux APIs is weird.

The main reason I did both of these was to show examples - in the last iteration of this proposal, @joshtriplett asked for a proof of concept that this could be applied to multiple (including Tier 1) platforms. I am not as familiar with macOS/Windows APIs for this so I chose Linux as the proof of concept target.

I think one advantage to using OS-specific trait extensions like this is that it would in theory allow platform experts for those particular platforms to add APIs as they saw fit, rather than shoehorning all platforms into a paradigm that doesn't make sense for them (e.g. enum-like priority values vs integer priority values).

See Processor Groups and Scheduling Priorities for Windows considerations.

Since (I assume) most libstd platforms support some form of setting priorities and/or affinities, surely there most be some hope of cross-platform APIs? Within the standard library I personally really prefer cross-platform APIs if at all possible because it provides the most value. Users get cross-platform support "for free" just by using normal APIs.

Thanks for the reference! This is interesting, actually:

All threads are created using THREAD_PRIORITY_NORMAL.

That seems to suggest to me that Windows doesn't actually provide a way to set priority before spawning a thread, which would mean that a true cross-platform API for Builder wouldn't be possible (maybe it could be a Result that always returns Err on Windows).

I definitely agree that cross-platform is ideal, but the scope of design for a fully cross-platform API for something like this seems fairly large. The idea with this proposal is to form the building blocks that could later be used to implement a cross platform API. I think by exposing these lower-level knobs we could enable third party crates like thread-priority to better experiment with APIs that could eventually be upstreamed into std, without committing to a specific design up front.

the8472 commented 1 year ago

Even with this mechanism though, I think OS-specific helpers would need to be provided to expose the underlying structures that control the spawning of the thread. Maybe it would end up looking something like this?

It means stuff doesn't have to live in std. E.g. if you have these 3 building blocks

  1. a thread-pool crate
  2. std's thread builder
  3. a crate to set priorities and affinities on running threads

Currently you can glue 1 and 2 together to give the thread pool named threads and custom stack sizes. But to set the pool's affinities you either have to hope for direct support from the pool crate or do some clunky dance that involves submitting a bunch of jobs to the pool which then set the affinity, those are difficult to coordinate.

If the builder gains closure-wrapping then these types suddenly compose better and you can use that crate to set affinities as soon as the thread starts running and you don't need affinity stuff in std.

It also avoids the need to have separate thread-create and running-thread APIs. Some of the former would have to be emulated with the latter anyway.

The only case where this doesn't work are platforms where these things must be set before thread startup.

joshtriplett commented 1 year ago

These extension traits should be sealed, so that we can extend them further in the future.

I do think we're going to need target-specific mechanisms here, because the same abstractions will not work for all targets. That said, I do wonder if we can provide some abstractions that work here. For instance, we could have the concept of "thread priority" in the form of:

pitaj commented 1 year ago

Agreed, I'm thinking we can at least have

impl OpaquePriority {
    /// Returns a new instance of the highest 
    /// possible thread priority. This may be equal 
    /// to default on platforms without thread priority.
    fn highest() -> Self;
    /// Returns a new instance of the 
    /// normal / default thread priority.
    // Maybe this should be a Default impl
    fn default() -> Self;
    /// Returns a new insurance of the lowest 
    /// possible thread priority. This may be equal 
    /// to default on platforms without thread priority.
    fn lowest() -> Self;
}

I also agree think that the target specific extension traits should come first, then the target-independent abstraction can be built upon it.

AzureMarker commented 1 year ago

Why are those on the thread-builder though? Thread priorities and affinities can be set after creation on most platforms.

At least for Horizon, the thread affinity can only be set before the thread is spawned.

ian-h-chamberlain commented 1 year ago

If the builder gains closure-wrapping then these types suddenly compose better and you can use that crate to set affinities as soon as the thread starts running and you don't need affinity stuff in std.

It also avoids the need to have separate thread-create and running-thread APIs. Some of the former would have to be emulated with the latter anyway.

The only case where this doesn't work are platforms where these things must be set before thread startup.

As @AzureMarker mentioned, one of the motivating platforms for this proposal indeed cannot set affinity after thread creation, which is why the proposed API operates on the Builder itself.

The closure-wrapping idea is interesting, but the more I think about it the more I wonder -- is there any advantage to this kind of API compared to what you can already accomplish with spawn()? Example:

std::thread::Builder::new()
    .spawn(|| {
        // some OS-specific API, possibly from 3rd party crate
        set_current_thread_priority(123);

        println!("hi");
    })
    .unwrap()
    .join()
    .unwrap();

// vs

std::thread::Builder::new()
    // possible new API:
    .pre_spawn(|| {
        // some OS-specific API, possibly from 3rd party crate
        set_current_thread_priority(123);
    })
    .spawn(|| {
        println!("hi");
    })
    .unwrap()
    .join()
    .unwrap();

The only thing I could think of offhand would be that the pre_spawn closure in this example could be FnMut() -> io::Result<()> or something like that, which could propagate errors from the pre-spawn closure to spawn() itself, perhaps? Or maybe I'm missing something in how I'm thinking about this kind of API?

the8472 commented 1 year ago

is there any advantage to this kind of API compared to what you can already accomplish with spawn()? Example:

Your example omits the interaction between different crates. If you have a threadpool crate which can be configured then one possible configuration item is the threadbuilder. Once you start submitting closures to the pool you have no control over on which thread the tasks will execute or how many closures will be executed on which thread. So a pool.submit() API wouldn't be the right place to set thread priorities. But a pool.init(builder) would work by passing a builder with a pre_spawn closure.

As @AzureMarker mentioned, one of the motivating platforms for this proposal indeed cannot set affinity after thread creation, which is why the proposed API operates on the Builder itself.

Sure, for those platforms it makes sense to implement extra methods because thread-spawning is encapsulated in std and not accessible to other crates. But for all other cases we can leave it to 3rd party code which can iterate on useful portable abstractions.

VorpalBlade commented 1 month ago

Maybe I'm missing something here, but the proposed Linux API in the initial post seems extremely lacking?

Setting CPU affinity is useful and needed, but so is scheduling policy and thread priority (speaking with my dayjob hat on, which involves real time Linux).

However, this is also very useful outside outside even the hard real time machine control crowd (that I belong to) though: rtkit-daemon, pipewire, several kernel threads and one kwin_wayland thread are all running with realtime priorities (a mix of SCHED_RR and SCHED_FIFO scheduling policies though, which does not seem optimal) on my bog standard Linux laptop.

And even if you don't care about the desktop audio/media use case, what about nice values? Those are also in use on my laptop, (though since I mostly work with real time myself I couldn't you tell you if this is actually generally useful or not).

Meziu commented 1 week ago

Maybe I'm missing something here, but the proposed Linux API in the initial post seems extremely lacking?

Setting CPU affinity is useful and needed, but so is scheduling policy and thread priority (speaking with my dayjob hat on, which involves real time Linux).

@VorpalBlade Just to clear any doubts, the proposed solution sketch API is meant to showcase the possible use cases of the extension trait BuilderExt, not its full implementation under Linux (or even Horizon). The actual implementation details and exposed APIs fundamentally depend on the platform they are being developed for, and it is up to those maintainers (not really us in this proposal) to make those changes.

However, this is also very useful outside outside even the hard real time machine control crowd (that I belong to) though: rtkit-daemon, pipewire, several kernel threads and one kwin_wayland thread are all running with realtime priorities (a mix of SCHED_RR and SCHED_FIFO scheduling policies though, which does not seem optimal) on my bog standard Linux laptop.

Indeed, which is why I find a bit weird (though understandable in most modern computing contexts) that the threading API exposed by std expects the OS scheduler to "do its thing" without any manual intervention. Expecting the thread to run under SCHED_OTHER is not a one-size-fit-all solution and, even though most platforms support switching scheduler at runtime, it should be possible to choose a more granular configuration before actually running the thread.

I'd like to see this proposal (or another solution to this problem) be discussed further, since I don't believe it should be up to third-party crates to handle such configurations when it is the job of the standard library to expose the threading API.