Unable to fit model when num_threads > number of logical cores on machine

[daniel-ferguson]

Hi, I ran into an issue around multithreading support when running goodbooks-recommender (https://maciejkula.github.io/2018/07/27/recommending-books-with-rust/)

When instantiating hyperparameters with num_threads greater than the number of cores on my CPU - the task never completes, and goodbooks-recommender's CPU usage drops to 0 almost immediately.

After some println!() driven debugging I narrowed down the issue to this section of sbr https://github.com/maciejkula/sbr-rs/blob/master/src/models/sequence_model.rs#L101-L168

I think the issue is caused by a combination of the following factors:

• work in the optimizer (from wyrm) is synchronized with barriers https://github.com/maciejkula/wyrm/blob/master/src/optim/mod.rs#L81
• rayon uses a threadpool to allocate work https://github.com/rayon-rs/rayon/blob/master/FAQ.md#how-does-rayon-balance-work-between-threads, and defaults to running n threads, where n is the number of logical cores https://github.com/rayon-rs/rayon/blob/master/FAQ.md#how-many-threads-will-rayon-spawn
• there will be m synchronized optimizers, where m is the number of threads specified https://github.com/maciejkula/wyrm/blob/12715ae99ca531db6557dca786e4a480ec608101/src/optim/mod.rs#L100-L102 and work cannot start until their barriers are synchronized
• unfortunately when num_threads is set higher than the number of logical cores, they will not all be synchronized, as rayon will only run n (number of cores) pieces of work at a time

---

Inserting a println!() before and after this line in wyrm (https://github.com/maciejkula/wyrm/blob/12715ae99ca531db6557dca786e4a480ec608101/src/optim/mod.rs#L81) illustrates this issue.

...
println!("pre barrier sync");
let _barrier = self.barrier_guard.synchronize();
println!("post barrier sync");
...

With thread count set to 4 all is fine and I see repeated "pre barrier sync", "post barrier sync" messages. If I set thread count to 5 I see 4 "pre barrier sync" messages followed by nothing, and the program hangs.

I'm not sure how to solve this, but hopefully this report is helpful nonetheless.

[maciejkula]

Sorry I didn't see this earlier!

I think you are quite right, both in that this is a bug and in terms of what causes it.

FWIW, the synchronized mode is primarily for unit tests: I don't think you would ever want to use it for real model fitting, the performance penalty is simply too large.