Closed ehiggs closed 8 years ago
When using PySpark Notebook and MLLib's ALS train function, we end up running out of memory. Maybe this is down to tuning, but looking around the web, I see that this is a known issue:
https://github.com/databricks/spark-perf/issues/92
Spark 1.6 is available which may help here. #146 has made this available.
This is an upstream issue; not with hod.
When using PySpark Notebook and MLLib's ALS train function, we end up running out of memory. Maybe this is down to tuning, but looking around the web, I see that this is a known issue:
https://github.com/databricks/spark-perf/issues/92