rayon-rs / rayon

Rayon: A data parallelism library for Rust
Apache License 2.0
10.94k stars 496 forks source link

Is it possible to spawn task to be executed in specific thread? #1199

Open AngelicosPhosphoros opened 5 days ago

AngelicosPhosphoros commented 5 days ago

I want to have something like broadcast but which allows to execute a task in a specific thread, using result of rayon::current_thread_index as a selector.

Context: it would allows to access something like bevy_ecs::NonSend soundly while running event loop entirely inside rayon threadpool.

cuviper commented 4 days ago

Do you really want any specific thread, or usually just the current thread? If it's the latter and that's blocking like broadcast, then you might as well just call whatever you wanted directly.

AngelicosPhosphoros commented 4 days ago

No, I want any specific thread. E.g. if I am running a game and have 2 non-send non-sync objects, e.g. native not-threadsafe Audio library in thread 2 and non-send Rhai VM in thread 3. I want to be able detect in thread 4 that there is nothing blocking execution of audio update and scripts update, so I spawn two tasks from thread 4 which must be executed exactly on thread 2 and thread 3, possibly in parallel to each other.

The reason why I want to do it this way instead of, e.g. execute non-sync parts on main thread and all other code in rayon, because I found out that if I put main event loop of a game into rayon threadpool, there is less jitter in FPS. Probably because time for waking main thread after finishing rayon tasks may vary wildly, and then there is need to wake rayon threads after starting new frame and submitting new tasks. When I run my main gameplay loop inside rayon thread pool, there is always at least one thread that never really blocks and keep working on tasks.

P.S. And I also want to have this thread-specific tasks to have bigger priority to normal tasks, because normal tasks can be stolen and executed by other threads while those are limited to one.

cuviper commented 4 days ago

No, I want any specific thread. E.g. if I am running a game and have 2 non-send non-sync objects, e.g. native not-threadsafe Audio library in thread 2 and non-send Rhai VM in thread 3. I want to be able detect in thread 4 that there is nothing blocking execution of audio update and scripts update, so I spawn two tasks from thread 4 which must be executed exactly on thread 2 and thread 3, possibly in parallel to each other.

Ok, and do you want to block (with work stealing) to wait for a return value like broadcast? Or fire-and-forget like spawn_broadcast? (possibly with a lifetime like Scope::spawn_broadcast)

The reason why I want to do it this way instead of, e.g. execute non-sync parts on main thread and all other code in rayon, because I found out that if I put main event loop of a game into rayon threadpool, there is less jitter in FPS. Probably because time for waking main thread after finishing rayon tasks may vary wildly, and then there is need to wake rayon threads after starting new frame and submitting new tasks. When I run my main gameplay loop inside rayon thread pool, there is always at least one thread that never really blocks and keep working on tasks.

Regarding the latency from the main thread, note that you can also make that act as part of the pool with ThreadPoolBuilder::use_current_thread.

P.S. And I also want to have this thread-specific tasks to have bigger priority to normal tasks, because normal tasks can be stolen and executed by other threads while those are limited to one.

That would be different than the current broadcasts, which are currently checked between local queue exhaustion and remote work stealing. Note that even if we add a new higher-priority queue that goes before normal tasks, we don't have any means of interrupting anything that's already executing, like an OS scheduler would with preemption. If you have that kind of need, it would probably be better to use dedicated threads outside of the pool.

AngelicosPhosphoros commented 4 days ago

Ok, and do you want to block (with work stealing) to wait for a return value like broadcast? Or fire-and-forget like spawn_broadcast? (possibly with a lifetime like Scope::spawn_broadcast)

I personally want Scope::spawn_broadcast for my use case.

That would be different than the current broadcasts, which are currently checked between local queue exhaustion and remote work stealing.

Well, it was something that nice to have but not crucial.