lyst / lightfm

A Python implementation of LightFM, a hybrid recommendation algorithm.
Apache License 2.0
4.77k stars 691 forks source link

Make LightFM supports CuPy sparse matrices #569

Open igorkf opened 4 years ago

igorkf commented 4 years ago

First of all, thank you very much for that awesome package!

I think lightFM could use CuPy to enable GPU computation.
I have a lot of data and making predictions take long time.
I'm already using cupy arrays to store the top-k recommended items, and I could decrease sorting the predictions using cp.argsort by the half of time! That's a huge improvement.

Now I'm thinking that the sparse matrices calculations could be massive calculations too.
How about support lightFM to use cupy sparse matrices?
https://docs.cupy.dev/en/stable/reference/sparse.html?highlight=sparse

dbalabka commented 2 years ago

I would reimplement the Cython part of the library using Numba or Jax to optimize for GPU and CPU at the same time.

Jax already has experimental sparse matrix support: https://jax.readthedocs.io/en/latest/jax.experimental.sparse.html