search

pishen / sbt-lighter

SBT plugin for Apache Spark on AWS EMR
Apache License 2.0
57 stars 15 forks source link

Adding support to pass both spark related args and job related args. #22

Closed alvinhenrick closed 6 years ago

alvinhenrick commented 6 years ago

Please feel free to make suggestion. I am adding support to pass spark related argument the number of executors , executor cores --con etc. The default existing behavior is maintained and not impacted.

I am looking for --sparkArgs argo arg1 ... --jobArgs argo arg1 ... If either of them do not exist it switches to default behaviors.

Thanks! Warm Regards, Alvin.

pishen commented 6 years ago

@alvinhenrick I think the first 4 commits are not related to this PR, could you reset your master first and let this PR only contain your latest change?

alvinhenrick commented 6 years ago

Done.

On Oct 13, 2017, at 5:56 PM, Pishen Tsai notifications@github.com wrote:

@alvinhenrick https://github.com/alvinhenrick I think the first 4 commits are not related to this PR, could you reset your master first and let this PR only contain your latest change?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pishen/sbt-emr-spark/pull/22#issuecomment-336598037, or mute the thread https://github.com/notifications/unsubscribe-auth/AC1BmxNWLzHLo8RCpNaxjEulyY1f0gVGks5ssAalgaJpZM4P5I4p.