nickgreenquist / cu2rec

CUDA Implementation of Parallel Matrix Factorization Algorithm for Recommender Systems
GNU Affero General Public License v3.0
13 stars 1 forks source link

Minibatch gradient descent #9

Open dorukkilitcioglu opened 6 years ago

dorukkilitcioglu commented 6 years ago

For now, we're doing a full gradient descent over the whole dataset (each thread does one user). That takes a lot of time, and increases the time till convergence. I think we can easily change this to minibatches using clever indexing, where we pass in a start and end indices, and only compute the gradients for those users. This will hopefully make the item factors (Q) converge faster, which would make the user factors (P) also converge faster.

Honestly, after coding it this way, I can kinda see the appeal of ALS.

dorukkilitcioglu commented 5 years ago

I don't think our current approach really works for any reasonably-sized dataset, and moving to minibatch gradient descent might make everything work. So this is now a priority.

dorukkilitcioglu commented 5 years ago

An implementation of this is in the minibatch branch. Until we know if this is even necessary, we're leaving it out.

dorukkilitcioglu commented 5 years ago

For the record, our issue in the second commit above ended up being something else. Still leaving this here, because it's an idea worth exploring.