search

snowplow / dataflow-runner

Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR
http://snowplowanalytics.com
19 stars 8 forks source link

Add EMR cluster resize command #48

Open jbeemster opened 5 years ago

jbeemster commented 5 years ago

This command would check whether the currently running cluster spec is the same as what is specified in the config. If not it would trigger a resize. This would let us manage our persistent clusters and before spec changes without having to terminate the whole system first.

alexanderdean commented 5 years ago

What are the semantics of an EMR resize - does it affect jobs running on the cluster?