Ensure that workers have Python dependencies installed

radanalyticsio / oshinko-s2i

This is a place to put s2i images and utilities for spark application builders for openshift

Apache License 2.0

15 stars 27 forks source link

Ensure that workers have Python dependencies installed #39

Closed willb closed 8 years ago

willb commented 8 years ago

MLLib in PySpark depends on NumPy, so we'll need to make sure the workers have access to it -- either via the openshift-spark image or via the pyspark image here (distributing NumPy to workers with the --py-files option to spark-submit).

tmckayus commented 8 years ago

Fixed in

https://github.com/radanalyticsio/oshinko-s2i/pull/40