radanalyticsio / oshinko-s2i

This is a place to put s2i images and utilities for spark application builders for openshift
Apache License 2.0
15 stars 27 forks source link

Custom Spark configuration is not performed when use OSHINKO_SPARK_DRIVER_CONFIG parameter when creating the app. #262

Closed ohonchar closed 6 years ago

ohonchar commented 6 years ago

During testing this tutorial https://radanalytics.io/howdoi/use-spark-configs, on step when try to specify OSHINKO_SPARK_DRIVER_CONFIG parameter, when creating the app, it specifying this parameter in app deployment configs, but does not performs them.

From other hand, if I use step whith manual creating cluster, useing command: "oshinko create mycluster --workerconfig=mysparkconfig --masterconfig=mysparkconfig", it deploys Spark clsters whith custom configuration without any problms.

tmckayus commented 6 years ago

This is working correctly, however we need to make the howdoi text more clear. The last example given there is meant to affect only the spark driver, not the spark cluster (that is, the master and worker nodes).

rebeccaSimmonds19 commented 6 years ago

Should this be moved to https://github.com/radanalyticsio/radanalyticsio.github.io? @elmiko @tmckayus ?

elmiko commented 6 years ago

@rebeccaSimmonds19 @ohonchar yes, it should be an issue against the site repo

rebeccaSimmonds19 commented 6 years ago

moved to radanalyticsio repo