search

snowplow / dataflow-runner

Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR
http://snowplowanalytics.com
19 stars 8 forks source link

Add jobflow name-based locking #30

Open alexanderdean opened 7 years ago

alexanderdean commented 7 years ago

This is a poor man's #17 - you use the EMR jobflow/cluster name to prevent multiple jobs from running at the same time. The jobflow will refuse to start if there is an existing running jobflow with the same name.

alexanderdean commented 7 years ago

This requirement was originally in the #20 ticket, but this morphed into file-based locking, so I've created this ticket for 0.4.0.

alexanderdean commented 6 years ago

Pushing back...