search

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.
Apache License 2.0
637 stars 116 forks source link

Hadoop / Spark conf templates #200

Open ncherel opened 7 years ago

ncherel commented 7 years ago

I really like Flintrock as I was able to completely set up my EC2 cluster with relative ease.

One thing that would be nice is to handle user templated configuration files for hadoop and spark. I was thinking of adding a field in config.yaml for the templates folder path.

If it sounds good to you, I can make a pull request.

nchammas commented 7 years ago

Ah, interesting. So you would provide your own templates using the keywords defined here? If so, then those template variables need to become part of Flintrock's public interface and documented somewhere.

Is that your intention?

ncherel commented 7 years ago

Yes exactly. Or maybe just export the template files to $HOME/.config/flintrock/templates/* so they can be modified freely. Add a flintrock templates --locate option to find them and maybe add a little more documentation about the mappings happening.