Lookup AWS instance types

snowch commented 2 years ago

With terraform it is possible to lookup instance types. This will allow deployment to continue (possibly at a higher cost) if the preferred instance type is not supported in the selected region and availability zone. E.g.

data "aws_ec2_instance_type_offering" "example" {
  filter {
    name   = "instance-type"
    values = ["t2.micro", "t3.micro"]
  }

  preferred_instance_types = ["t3.micro", "t2.micro"]
}

Source:

Also, aws_ec2_instance_types. E.g.

data "aws_ec2_instance_types" "test" {
  filter {
    name   = "auto-recovery-supported"
    values = ["true"]
  }

  filter {
    name   = "network-info.encryption-in-transit-supported"
    values = ["true"]
  }

  filter {
    name   = "instance-storage-supported"
    values = ["true"]
  }

  filter {
    name   = "instance-type"
    values = ["g5.2xlarge", "g5.4xlarge"]
  }
}

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ec2_instance_types

erdincka commented 2 years ago

This would be great, especially considering that some instance types are limited (such as GPU nodes) in certain regions and not always available. Though, is it possible that the same AMIs not working for some of these alternative instance types? If not, I believe we should implement this right away (to extend to more regions and support more users/projects in the most common regions).

snowch commented 2 years ago

I've been trying different regions and I've been hoping to hit a failure due to unsupported instance types but I haven't run into one yet, though I've only tried a few regions. I'll keep pushing on this...

snowch commented 2 years ago

notes:

TYP="m5.xlarge"

REGIONS=$(aws ec2 describe-regions \
    --all-regions \
    --query "Regions[].{Name:RegionName}" \
    --output text)

for REGION in $REGIONS;
do
   echo $REGION
   aws ec2 describe-reserved-instances-offerings --region $REGION --filters 'Name=scope,Values=Availability Zone' --no-include-marketplace --instance-type $TYP | jq -r '.ReservedInstancesOfferings[].AvailabilityZone' | sort | uniq
done

erdincka commented 2 years ago

Within the container;

root@127675acc96d:/app/server# REGIONS=$(aws ec2 describe-regions \
    --all-regions \
    --query "Regions[].{Name:RegionName}" \
    --output text)

TYP="m5.xlarge"

for REGION in $REGIONS;
do
   echo $REGION
   aws ec2 describe-reserved-instances-offerings --region $REGION --filters 'Name=scope,Values=Availability Zone' --no-include-marketplace --instance-type $TYP | jq -r '.ReservedInstancesOfferings[].AvailabilityZone' | sort | uniq
done
af-south-1

An error occurred (AuthFailure) when calling the DescribeReservedInstancesOfferings operation: AWS was not able to validate the provided access credentials
eu-north-1
eu-north-1a
eu-north-1b
eu-north-1c
ap-south-1
ap-south-1a
ap-south-1b
ap-south-1c
eu-west-3
eu-west-3a
eu-west-3b
eu-west-3c
eu-west-2
eu-west-2a
eu-west-2b
eu-west-2c
eu-south-1

An error occurred (AuthFailure) when calling the DescribeReservedInstancesOfferings operation: AWS was not able to validate the provided access credentials
eu-west-1
eu-west-1a
eu-west-1b
eu-west-1c
ap-northeast-3
ap-northeast-3a
ap-northeast-3b
ap-northeast-3c
ap-northeast-2
ap-northeast-2a
ap-northeast-2b
ap-northeast-2c
ap-northeast-2d
me-south-1

An error occurred (AuthFailure) when calling the DescribeReservedInstancesOfferings operation: AWS was not able to validate the provided access credentials
ap-northeast-1
ap-northeast-1a
ap-northeast-1c
ap-northeast-1d
sa-east-1
sa-east-1a
sa-east-1b
sa-east-1c
ca-central-1
ca-central-1a
ca-central-1b
ca-central-1d
ap-east-1

An error occurred (AuthFailure) when calling the DescribeReservedInstancesOfferings operation: AWS was not able to validate the provided access credentials
ap-southeast-1
ap-southeast-1a
ap-southeast-1b
ap-southeast-1c
ap-southeast-2
ap-southeast-2a
ap-southeast-2b
ap-southeast-2c
eu-central-1
eu-central-1a
eu-central-1b
eu-central-1c
ap-southeast-3

An error occurred (AuthFailure) when calling the DescribeReservedInstancesOfferings operation: AWS was not able to validate the provided access credentials
us-east-1
us-east-1a
us-east-1b
us-east-1c
us-east-1d
us-east-1f
us-east-2
us-east-2a
us-east-2b
us-east-2c
us-west-1
us-west-1a
us-west-1b
us-west-2
us-west-2a
us-west-2b
us-west-2c
us-west-2d
root@127675acc96d:/app/server#

erdincka commented 2 years ago

Is this normal/expected?

snowch commented 2 years ago

It looks more interesting with TYP="p4d.24xlarge"

eu-north-1
ap-south-1
eu-west-3
eu-west-2
...
eu-west-1
eu-west-1c
ap-northeast-3
...

For the regions displayed above, I would say that p4d.24xlarge is only available in eu-west-1, in the AZ c

hpe-container-platform-community / ezdemo

Lookup AWS instance types #10