Closed terrytangyuan closed 8 years ago
This sparkit-learn package recently came out that utilizes Spark to scale scikit-learn and aid GridSearch. https://github.com/lensacom/sparkit-learn
Since skflow has a sklearn-like interface, it might be a good idea to show an example of using sparkit-learn to scale skflow with minor changes.
Blogpost can be found here: https://databricks.com/blog/2016/02/08/auto-scaling-scikit-learn-with-spark.html
PRs are welcomed for this.
@terrytangyuan Can you move this FR tensorflow/tensorflow with may be updated details about existing libraries around TF and Spark?
Closing now since someone submitted an issue at sparkit-learn repo.
This sparkit-learn package recently came out that utilizes Spark to scale scikit-learn and aid GridSearch. https://github.com/lensacom/sparkit-learn
Since skflow has a sklearn-like interface, it might be a good idea to show an example of using sparkit-learn to scale skflow with minor changes.
Blogpost can be found here: https://databricks.com/blog/2016/02/08/auto-scaling-scikit-learn-with-spark.html
PRs are welcomed for this.