search

pishen / sbt-lighter

SBT plugin for Apache Spark on AWS EMR
Apache License 2.0
57 stars 15 forks source link

Would be nice to expose some decorators in settings #8

Closed alexmnyc closed 7 years ago

alexmnyc commented 7 years ago

For example, I need to add withEc2KeyName so that I can get onto the cluster. It would be nice to have a generic way of adding additional config attributes via some decorators similar to:

extraJobFlowInstancesConfig := { (jobFlow:JobFlowInstancesConfig) => jobFlow.withEc2KeyName("MyKeyName") }

pishen commented 7 years ago

Tried to fix this in version 0.8.0, which expose the underlying AWS objects for you to modify the settings. Please refer to https://github.com/pishen/sbt-emr-spark/tree/0.8.0#modify-the-configurations-of-underlying-aws-objects for more details.