Expose thread/task priorities

[steveklabnik]

Issue by hoeppnertill Wednesday Jul 23, 2014 at 18:09 GMT

For earlier discussion, see https://github.com/rust-lang/rust/issues/15927

This issue was labelled with: A-libs in the Rust repository

---

Trying to find an equivalent to the following C++ code,

struct sched_param p;
p.sched_priority = priority;
pthread_setschedparam(thread_, SCHED_FIFO, &p);

I failed to set the task priority using just exposed API.

brsons statement on IRC,

you can just call the pthread functions, but you'll be creating a thread that doesn't have the runtime setup properly

leaves little doubt that an exposed API for this actually is needed.

[jonhoo]

It would be good if this also included exposing sched_getaffinity and sched_setaffinity, as these could be useful for things like determining the number of CPUs.

[nagisa]

Windows

[steveklabnik]

https://news.ycombinator.com/item?id=15518320

[jq-rs]

I think this is a worthy improvement to have as some systems do use priorities heavily.

[rayascott-zz]

This would be extremely useful for situations where you’re wanting to shift low priority workloads off to queues that run in the background. #trifecta

[buzmeg]

This crate seems to fit the bill for this RFC: https://docs.rs/thread-priority/0.2.0/thread_priority/index.html

Is it possible to get a pronouncement on it? Thanks.

[naturallymitchell]

Thread management is a complex task. https://github.com/rust-lang/rfcs/issues/613 established the direction for concurrency tooling with Actix. Then to reliably and effectively specify domain specific logic, such as scheduling and priority, a higher level scripting layer is needed. Torchbear has an issue to support cooperative multitasking for Speakeasy's collaborative runtime language, https://github.com/speakeasy-engine/torchbear/issues/74

An example of stream relations mapping, ie with dependency and weighting, is already available in Cloudflare's Adopting a new Approach to HTTP Prioritization. This is one very helpful component that'll make this messy implementation process clean.

[buzmeg]

Quoting upstream:

you can just call the pthread functions, but you'll be creating a thread that doesn't have the runtime setup properly

Now, it seems that looking at some of the external comments, this statement isn't actually true anymore.

It looks like Rust has changed in some way since 2015 and the standard pthreads mechanisms now work. If that is actually the case, then this should probably be closed with "Use pthreads library calls".

Can someone with internals knowledge verify this?

[dljsjr]

I made the comment that @steveklabnik linked here: https://github.com/rust-lang/rfcs/issues/819#issuecomment-338308761

@buzmeg that crate gets close but it doesn't quite cut it. For one, on Linux, musl and libc handle the scheduler APIs differently. Musl attempts to be fully POSIX adherent and the API extensions added by the low-latency/fully preemptible Linux patches are not. For example: https://www.openwall.com/lists/musl/2016/03/01/5. I've already tried to deploy software using that crate in a musl based environment and it fails.

Additionally, I don't believe it handles Mach threads on macOS, which don't use the pthread API for thread priorities at all. I implemented a real-time thread scheduling utility library on macOS a few years back at another gig and it is very different from POSIX.

I can't comment on Windows.

@jonhoo bits on affinity and such might also be impacted by API differences in musl and libc, mach, etc.

When I originally made that comment I was using Rust as a hobbyist. I now write Rust at my day job (and am in fact in a place where I need to do real-time threading, core pinning, etc in an embedded environment with an RTOS) and I just wanted to chime in to say that it would still be super cool if a cross-platform implementation of this existed in the core or std, especially because of all the various skews between the tier 1 targets.

[naturallymitchell]

Handling task priority, scheduling, and queuing in OS threads is too low-level. Rust with Actix can do this, similar to how Scala handles Akka actor priorities. Programming "actors" requires an additional layer of logic though, which is why I also mentioned Torchbear for its higher-level scripting environment. You'll see, when programming domain logic priority, as well as other higher dimensional attributes, the tools built on Torchbear are Godsent.

[naturallymitchell]

I need to do real-time threading, core pinning, etc in an embedded environment with an RTOS

I haven't used core pinning yet, but Actix seems to use pinning extensively to safely track future readiness for actors.

[dljsjr]

@naturallymitchell The Pin and Unpin traits are use to establish that objects stay in the same place in memory. Core pinning is very different, it's is for hinting to the OS's scheduler that a thread of execution should not be moved to a different physical core during context switches. This is important for situations where context switching overhead is undesired or you want to maximize cache locality.

An actor library would be nice but I don't think that belongs in the std/core necessarily, channels get you most of the way there anyway for small use-cases. Regardless, this issue isn't about tracking development of high level concurrency, it's about exposing existing APIs for low-level threading to the Rust ecosystem without needing to use unsafe/C bindings from third-party crates. These are needed tools when working in e.g. real-time computing where deterministic latency and execution time are more important than throughput/performance.. Without being able to elevate the your thread priority as the OS level to a certain threshold, then your work will still be pre-emptible by the operating system or kernel drivers or the like. Sometimes threads need to be un-preemptible and since Rust is a systems language it would be a good feature to have implemented in a well-maintained and safe crate, IMO.

[naturallymitchell]

@dljsjr Oh okay! Thank you for clarifying about Pin and Unpin for OS level thread management. Since POSIX and Windows API have already been explored a bit, I believe it would make sense for us to look into how Google's Zircon kernel scheduling works. It appears to provide an improved approach to thread scheduling. What do you think then about adding interfaces in Rust for Fuchsia's processor resource management?

[comex]

Additionally, I don't believe it handles Mach threads on macOS, which don't use the pthread API for thread priorities at all. I implemented a real-time thread scheduling utility library on macOS a few years back at another gig and it is very different from POSIX.

macOS does implement pthread priority APIs (on top of the Mach ones); it just doesn't support putting threads into realtime mode that way.

[dljsjr]

@comex good to know, I only ever used the Mach task ports directly when I had to do that sort of thing on macOS.

[naturallymitchell]

I spoke with the maintainer of Zircon on Freenode #fuchsia https://freenode.logbot.info/fuchsia/20200706#c4313960-c4314889 https://freenode.logbot.info/fuchsia/20200707

Fuchsia is even better than as documented

Rust bindings already exist!

Who's gonna try this?!