Closed ckingdev closed 5 years ago
Most of these algos rely on numpy matrix operations, which in turn rely on BLAS, which is typically multi-threaded. So even if there's no explicit parallelization logic, you should see an improvement on multi-core machines. It's definitely the case with RP, but I'm not sure about HDP, you may be right.
In the future, it's better to ask the mailing list and discuss there.
I'm well aware that BLAS is used here. You and I both know that that's not what people are going to read "multicore HDP" as. Especially since the package does have a parallel implementation of LDA.
@ckingdev are you a user of HDP? I'm actually considering dropping that model from Gensim altogether.
Some words on where/how HDP is useful to you, a workflow from an actual user, could help us make that decision. Cheers!
The readme currently (on master) states:
However, the HDP model is single core (and it looks like RP is as well, though I'm less familiar with the technique) and this is misleading.