Generate spark configs for vanilla hadoop jobs

hpcugent / hanythingondemand

hanythingondemand provides a set of scripts to easily set up an ad-hoc Hadoop cluster through PBS jobs

https://hod.readthedocs.org

GNU General Public License v2.0

12 stars 6 forks source link

Generate spark configs for vanilla hadoop jobs #150

Open ehiggs opened 8 years ago

ehiggs commented 8 years ago

We've had a request that we generate spark-defaults.conf for Hadoop jobs. This just means moving the autogen code for Spark from hod.config.autogen.ipython_notebook.py to hod.config.autogen.hadoop.py.