EKS-Optimized AMI Metadata SSM Parameter

tabern commented 5 years ago

Tell us about your request This feature will make metadata about EKS-optimized AMIs available as an AWS SSM Parameter. You can use the SSM API to retrieve the latest optimized AMI IDs programmatically.

Example (proposed) API call using the AWS CLI: aws ssm get-parameters --names /aws/service/eks/optimized-ami/amazon-linux/recommended

Which service(s) is this request for? Amazon EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? Today, you have to lookup the latest AMI IDs via the EKS documentation in order to keep your templates for launching EKS worker nodes up to date.

d-nishi commented 5 years ago

@tabern @mcrute any timeline available for this feature implementation?

Thanks!

pawelprazak commented 5 years ago

@tabern would this help?

aws ec2 describe-images \
        --profile $(AWS_PROFILE) \
        --region $(AWS_REGION) \
        --owners 602401143452 \
        --filters \
            "Name=virtualization-type,Values=hvm" \
            "Name=architecture,Values=x86_64" \
            "Name=root-device-type,Values=ebs" \
            "Name=state,Values=available" \
            "Name=name,Values=amazon-eks-node-*" \
        --query 'Images | sort_by(@, &CreationDate)| [-1].ImageId' \
        --output text

metral commented 5 years ago

It'd be great to know: is the target AMI planned to be returned here actually going to be the latest, patched and suggested version one should be using with EKS?

This ask tends to be a distinction from sorting only based on the creationDate of the AMI's because it seems that many images can be published on the same day, and at different times, which could not always translate to the absolute latest version available.

tabern commented 5 years ago

@metral that is correct. Returning the AMI through the SSM parameter will be equivalent to what we publish today in the EKS documentation

metral commented 5 years ago

I can't stress how important it is to have a programmatic way to retrieve the latest supported AMIs that work for both control plane AND worker nodes together.

With the release of the 1.13 worker AMI this week, using the latest AMI sorted by CreationDate for the workers will return the 1.13 AMI, when the EKS clusters console has yet to be updated to v1.13 and is currently (as of 06/19/19) still stuck on v1.12:

This creates a version skew of a control plane on v1.12 and kubelets on v1.13 and is not supported. This mismatch leads to a heap of errors for the CNI config not being initialized on the workers and them remaining in a NotReady state, as seen here.

This same mismatch happened last time when the 1.12 worker AMI was cut, but the control plane only supported versions up to 1.11. It eventually subsided when the control plane options were updated thereafter, but nevertheless, in this time period using AMIs by CreationDate / latest leads to broken clusters.

When can we expect an update on https://github.com/aws/containers-roadmap/projects/1#card-19530403?

metral commented 5 years ago

EKS console has now made 1.13 available for the control plane, but 1.12 is still the default.

If possible, until this SSM API is available, can we consider doing a tandem release of both control plane versions supported in the console + worker Node AMIs available to ensure version skew can't happen for smart default selectors based on CreationDate?

FWIW, we (@pulumi) use this CreationDate format, as does and eksctl.

tabern commented 5 years ago

Hey @metral thanks for the feedback - this feature (SSM parameter) is pretty important to us and were working hard to enable it.

Agree - having clusters trying to create worker nodes that are a version ahead of the control plane is not a good pattern. The console update has been a bit slow to roll out, especially for 1.13 and we're working to fix that in the future so we have a single, global coordinated launch across API, AMI, and consoles.

swibrow commented 5 years ago

Why not add the eks version in the ami lookup query?

webframp commented 5 years ago

It appears this is now available: https://aws.amazon.com/about-aws/whats-new/2019/09/amazon-eks-provides-eks-optimized-ami-metadata-via-ssm-parameters/

mikestef9 commented 5 years ago

Amazon EKS now supports retrieving EKS-Optimized AMI IDs via SSM parameters.

You can query the SSM Parameter Store API using the AWS CLI, any of the AWS SDKs, or directly from CloudFormation templates.

Note AMI IDs are no longer directly published in EKS documentation. There are links to follow on this page to the Systems Manager Console where you will find the AMI IDs

Links

aws / containers-roadmap

EKS-Optimized AMI Metadata SSM Parameter #231