Synchronous nature of join might result in suboptimal use of compute resources

[zopsicle]

The documentation for join reads:

…; if however closure B has been stolen, then it will look for other work while waiting for the thief to fully execute closure B.

Even if closure B completes before the “other work” completes, the code following the call to join cannot run immediately after closure A and closure B complete, even if there is a free thread on the thread pool. It must wait for the “other work” to complete, which might take a long time compared to closures A and B, with no way to prefer certain tasks to be considered “other work” candidates (AFAIK).

This is only really an issue because join must return on the calling thread, by its synchronous nature. Having an async version of join (i.e. one that returns a future) would improve on this by allowing it to return on a different thread than the one it was called on, because the code before and after the await are separate tasks. It would require running futures on the thread pool.

Is this something that has been explored before?

[cuviper]

Asynchronous code is quite different from the user perspective, compared to Fn traits. It's not something we can just graft in automatically, but there have been experiments with futures if you search through issues. Alternatively, if you're concerned about such latency, have you considered using tokio's thread pool?

[godmar]

tokio's thread pool?

I too am interested in this question. I don't believe tokio's thread pool has the functionality one would expect here (based on a quick reading of the documentation of Future). In particular, if you join a task in Tokio that hasn't been started, the calling thread won't take on this task and run it inline (I believe.)

I was hoping Rayon would provide something similar to Java's Fork/Join where futures are first-class objects that can be later joined. This would make it ergonomically possible to easily spawn multiple tasks before joining any, rather than just two as provided by rayon::join.

Also, a side note; it seems Rayon lets workers blocked on a stolen task steal from any other worker (based on this comment); experience with FJP has shown that it's usually best to steal only from the worker that stole the task. Otherwise, a worker may easily take on too many tasks, and because Rayon is also stack-based and doesn't support child stealing the way Cilk does, would then be committed to these tasks and become a straggler later on when other threads are waiting for it to finish. I don't know if this has been observed in Rayon as well.

[cuviper]

experience with FJP has shown that it's usually best to steal only from the worker that stole the task.

That sounds interesting -- do you have any references?

[godmar]

experience with FJP has shown that it's usually best to steal only from the worker that stole the task.

That sounds interesting -- do you have any references?

Personal communication with Doug Lea. But you may find discussions on the interest mailing list.

ps: actually, it was also personal observation (not in Java, but in another threadpool I implemented.) I asked Doug, and he said he'd noticed the same thing and that's why FJP steals only from the thief.