search

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.
Apache License 2.0
636 stars 116 forks source link

Gracefully handle interrupted launches: allow flintrock caller to more easily idenify instances #339

Closed rlaabs closed 9 months ago

rlaabs commented 3 years ago

I typically call flintrock as a subprocess, sometimes (eg when the wait for spot requests is too long), the subprocess times out. If this happens before the tags are set in ec2.launch() it is difficult to identify the instances that need to be cleaned up.

Would you be open to a PR that sets the user defined tags in the earlier ec2._create_instances() call?

I'm also working on a PR for the interrupted launch related #183 , but it may be best to split this one out?

nchammas commented 3 years ago

Oh neat. I see that the EC2 API added this ability in 2017, a couple of years after I wrote the initial cuts of launch() and _create_instances().

I agree, it's better to add the tags as part of the operation that creates the instances, as opposed to in a separate step. 👍