dtreduce

[orenbenkiki]

I just opened https://github.com/JuliaLang/Distributed.jl/issues/67 to ask for better Julia support for a scenario where one has multiple worker processes, where each has multiple threads.

In such a scenario, it would be useful to have a dtreduce function which works across all the threads across all the worker processes. This would be different from dreduce which only uses one thread in each worker process.

It would probably be advisable to keep the current behavior of dreduce to keep it compatible with @distributed and pmap. Keeping the ability to have one per-process reducers is usefull, for example if the invoke @threads internally.

[tkf]

This is already implemented in JuliaLang/julia#133. It'll be in the next release. You can pass threads_basesize = typemax(Int) if you don't want to use threads.

Keeping the ability to have one per-process reducers is usefull, for example if the invoke @threads internally.

This is not necessary because Julia's scheduler is depth-first: https://julialang.org/blog/2019/07/multithreading If you use @threads inside (e.g.) f of Map(f) then it will automatically use multiple cores.

Though this may be a good idea if you use external libraries that are not aware of Julia's scheduler. https://github.com/JuliaLang/julia/issues/32786