quinngroup / dr1dl-pyspark

Dictionary Learning in PySpark
Apache License 2.0
1 stars 1 forks source link

Distributed ALS implementations #69

Open magsol opened 8 years ago

magsol commented 8 years ago

Here's how Spark's MLlib implements distributed ALS. Worth looking into: https://databricks-training.s3.amazonaws.com/movie-recommendation-with-mllib.html

magsol commented 8 years ago

With respect to u and v, how do these vectors change over the course of the 1D ALS step? I ask from the perspective of the following idea: are there portions of the vectors that stop changing during the ALS step? If so, this is a potential point of optimization: when one of the elements stops changing, we ignore it in future iterations, concentrating only on iterating over the elements that are changing until they stop moving.