snowplow / dataflow-runner

Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR
http://snowplowanalytics.com
19 stars 8 forks source link

Add ability to set a SecurityConfiguration in RunJobFlowInput #78

Closed jbeemster closed 2 years ago

jbeemster commented 2 years ago

Currently it is not possible to specify a custom SecurityConfiguration which means it is not possible to disable things like IMDSv1 on EMR to enhance the security of the system.

In this type: https://docs.aws.amazon.com/sdk-for-go/api/service/emr/#RunJobFlowInput Add: SecurityConfiguration

The option needs to be mapped into this object: https://github.com/snowplow/dataflow-runner/blob/master/src/emr_cluster.go#L186-L197


Resources: https://aws.amazon.com/about-aws/whats-new/2021/03/amazon-emr-now-supports-amazon-ec2-instance-metadata-service-v2/