The possibility of dynamic add/remove threads as need in threadpool?

[Fomalhauthmj]

Assuming I have built a thread pool with n threads.I will spawn n tasks, which is a loop of processing computation-heavy works.These tasks continuously processing until received the finished signal of all works.

Because creating working environment is expensive, i don't want to create task for each work. Importantly,some task may block due to work.I think it's a waste on the cpu cycles of blocking thread.

So I want to dynamic add thread when have blocking thread,and remove redundant thread when blocking thread resumes,just as the docs of ThreadPoolBuilder::num_threads .

Do we have space to implement this behaviour or any available suggestions?

[cuviper]

We documented the possibility of dynamic thread counts as a reservation for future enhancement, but I think it would be hard in practice. There are a number of places across the thread pool that deal with the count which we would have to sync up -- probably a "stop the world" kind of event.

For your case, would it work if there was a way to move that blocking operation to a new thread? Then we would allow the current thread to do other work. Presumably you'd want that to join back eventually though, so I think we would need a new rayon-core primitive for external blocking that does work-stealing in the meantime.

[Fomalhauthmj]

We documented the possibility of dynamic thread counts as a reservation for future enhancement, but I think it would be hard in practice. There are a number of places across the thread pool that deal with the count which we would have to sync up -- probably a "stop the world" kind of event.

For your case, would it work if there was a way to move that blocking operation to a new thread? Then we would allow the current thread to do other work. Presumably you'd want that to join back eventually though, so I think we would need a new rayon-core primitive for external blocking that does thread-stealing in the meantime.

I think moving blocking operation to a new thread is hard in my case.Specifically,my rayon thread is working at stack-based opcode computation(blockchain vm).When they read some dependent value,they will block until they can successfully read again( wait custom condvar).So,move preceding computation work to another thread for me is unknown and tough.

Because my sync mechanisms between threads is global scheduler and states reference,so this new thread don't need to join back.And,what's the meaning of external blocking that does thread-stealing in the meantime?

[cuviper]

And,what's the meaning of external blocking that does thread-stealing in the meantime?

Sorry, I meant to say "work-stealing" -- updated. The idea is that we could have something that blocks, but still does other work on the current thread until that result is ready. This is essentially what you get with async and Future::poll, and maybe we could have a rayon::block_on(impl Future) function like that. That may not be easy though.

[ZhennanWu]

I happen to run into a similar situation where some work units would trigger expensive blocking synchronization, and the work units are organized in a tree structure with parent awaiting children's scopes (So, I don't see a way of migrating the work into a new thread). I ended up by pessimistically separate potential synchronization participants to eliminate the synchronization at the loss of parallelism.

If something like rayon::block_on(impl Future) is one day implemented, then that means there would also be rayon::Mutex rayon::RwLock, which is quite a big change to rayon's programming model.