[Enhancement]: Support for Custom AMI ID per Instance in InstanceTypeConfigs Block of Instance Fleet

[Arullaldivakar]

Description

AWS recently introduced support for specifying a CustomAmiId per instance type within an InstanceTypeConfigs block for an InstanceFleet. This enhancement provides the flexibility to use different AMIs with different EC2 instance types within a single instance fleet, which is particularly useful for diverse workloads that may require different configurations or software packages.

AWS Support Example:

Here's an example of how AWS supports this feature:

bash
aws emr add-instance-fleet --cluster-id j-123456 --release-label Amazon EMR 5.3.1 \
  --service-role EMR_DefaultRole \
  --ec2-attributes InstanceProfile=EMR_EC2_DefaultRole \
  --instance-fleet \
    InstanceFleetType=Task,TargetSpotCapacity=1,\
InstanceTypeConfigs=['{InstanceType=m5.xlarge,CustomAmiId=ami-123456}',\
'{InstanceType=m6g.xlarge,CustomAmiId=ami-234567}']

Affected Resource(s) and/or Data Source(s)

• aws_emr_cluster['core_instance_fleet']
• aws_emr_instance_fleet

Potential Terraform Configuration

resource "aws_emr_instance_fleet" "task" {
  cluster_id = aws_emr_cluster.cluster.id

  instance_type_configs {
    bid_price_as_percentage_of_on_demand_price = 100
    ebs_config {
      size                 = 100
      type                 = "gp2"
      volumes_per_instance = 1
    }
    instance_type     = "m4.xlarge" 
    weighted_capacity = 1
    custom_ami_id     = "ami-123456"  # <-- Desired Addition
  }

  instance_type_configs {
    bid_price_as_percentage_of_on_demand_price = 100
    ebs_config {
      size                 = 100
      type                 = "gp2"
      volumes_per_instance = 1
    }
    instance_type     = "m4.2xlarge"
    weighted_capacity = 2
    custom_ami_id     = "ami-234567"  # <-- Desired Addition
  }

  launch_specifications {
    spot_specification {
      allocation_strategy      = "capacity-optimized"
      block_duration_minutes   = 0
      timeout_action           = "TERMINATE_CLUSTER"
      timeout_duration_minutes = 10
    }
  }

  name                      = "task fleet"
  target_on_demand_capacity = 1
  target_spot_capacity      = 1
}

References

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-fleet.html#:~:text=Example%3A%20Add%20a%20task%20node%20to%20a%20running%20cluster%20with%20multiple%20instance%20types%20and%20multiple%20custom%20AMIs

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[trutled3]

I am interested in working on this enhancement. I will open a PR to add this capability.