Yelp / mrjob

Run MapReduce jobs on Hadoop or Amazon Web Services
http://packages.python.org/mrjob/
Other
2.62k stars 586 forks source link

Assign tags on EMR creation in single API call #2207

Open mgmarino opened 3 years ago

mgmarino commented 3 years ago

AWS has deprecated several managed EMR policies (see here) and the replacement policies make use of tags for authorization.

Currently, tags are assigned after a cluster is made, in a separate api call, though the underlying boto3 client supports setting tags on cluster creation.

To avoid any issues that may arise with failed tag application - either due to a subsequent failure of the tags call, or because of the (albeit small) time between API calls - the tags should be assigned in the same API call as the cluster creation.