Add settings to submit multiple EMR steps

pishen / sbt-lighter

SBT plugin for Apache Spark on AWS EMR

Apache License 2.0

57 stars 15 forks source link

Closed SrikanthTati closed 7 years ago

SrikanthTati commented 7 years ago

Add new setting sparkEMRSteps and tasks sparkSubmitSteps & sparkUploadJarToS3 to add multiple steps on a EMR cluster.

Also added setting for autoscaling role and key file name for SSH access.

SrikanthTati commented 7 years ago

Any thoughts on this?

pishen commented 7 years ago

@SrikanthTati Sorry, will give it a look this weekend :P

pishen commented 7 years ago

@SrikanthTati I have written some comments on this. Let's see if these make sense for you? :P

pishen commented 7 years ago

Thanks for the contribution!