Open meetreks opened 1 year ago
We support custom AMIs through the amiSelector
in the AWSNodeTemplate
. Could you tag the AMIs that you want to resolve through SSM alias?
Hi @jonathan-innis Thanks for reverting back. Couple of things, 1. As SSM Aliases are currently supported - albeit for default AMI families, would it make sense to support custom AMI families as well?
@meetreks This is something that we are going to take into consideration as part of #1327. We're still not sure if SSM aliases are the right way to go, though. Ideally, we would like to support a default tagging mechanism for the default AMIs such that amiSelector mechanism could work for both custom AMIs and default AMIs.
Which AMIs are you trying to resolve through SSM?
@jonathan-innis Many thanks for reverting back. We create custom EKS AMI's to cater for different use cases - plain-vanila, with lustre, etc..These are the custom AMI(s) which we would like to be handled via SSM alias to keep it inline with Terraform and other method possible today.
keep it inline with Terraform and other method possible today
Is it possible to use tags in this case or change your terraform to also use tags. Could you tell me what the operational burden/overhead looks like for leveraging tags vs. leveraging SSM in your image deployment pipelines?
There's Custom
amiFamily already, so I think we can use implement that
There's
Custom
amiFamily already, so I think we can use that
Its dummy code, check it out.
I mean we can implement on that Custom amiFamily
keep it inline with Terraform and other method possible today Is it possible to use tags in this case or change your terraform to also use tags. Could you tell me what the operational burden/overhead looks like for leveraging tags vs. leveraging SSM in your image deployment pipelines?
This feature ensures that all our AMIs to be up to date, our enterprise policy means we have frequent AMI updates and Tenants/Clients are not always looking to update them as priority, this feature helps to automate this process using in conjunction with Drift functionality. It also reduces the need of any bespoke code to implement this on our side using for ex.. events, pipelines triggers and etc. And also, this feature is something that is supported on terraform side and hence its readily available with MNG with ASG , so we feel that Karpenter should also match this feature.
this feature is something that is supported on terraform side and hence its readily available with MNG with ASG
I'm mainly curious what would be the operational burden to add a specific tag to AMIs that Karpenter could use and what the operation burden of this might look like to use it on MNG and ASG.
I'm open to this feature add, I'm mainly just curious how much extra work the current state of the world is so that we can better understand the ask
this feature is something that is supported on terraform side and hence its readily available with MNG with ASG
I'm mainly curious what would be the operational burden to add a specific tag to AMIs that Karpenter could use and what the operation burden of this might look like to use it on MNG and ASG.
I'm open to this feature add, I'm mainly just curious how much extra work the current state of the world is so that we can better understand the ask
The AMI vending is a separate process and is rigid in terms of adding custom tags for tennants and hence the request to keep it similar to the other options available today.
Picked up this issue, will advise when completed.
@meetreks Hey how's it going? Feel free to reach out to us in the kubernetes slack if you have questions.
@njtran just got to work on this, will ping you folks in slack for sure.
Just to add to this ticket – tags-on-AMI is not a usable solution if you use an AMI which is shared from another account. This is because AWS AMI sharing (and RAM in general) does not propagate the tags to the shared account.
Supporting ParameterStore natively is much preferred.
Tags on AMIs seem an awkward way to indicate the AMI choice.
It makes more sense to simply store a reference to the chosen AMI (either in an SSM parameter, or a singular Kubernetes resource spec, or a tag attached to the EKS cluster rather than to the AMI). That way the updates are inherently atomic, and different clusters (e.g. prod vs staging) can implement different selections concerning the same range of AMIs.
Tell us about your request
Hi Team,
As per the code here https://github.com/aws/karpenter/blob/main/pkg/providers/amifamily/ami.go#L135 , SSM Alias for AMI seems to be handled but I understand speaking the the engineer that this is default SSM alias and hence custom SSM alias is not handled (yet). The documentation for this is available here -- https://docs.aws.amazon.com/autoscaling/ec2/userguide/using-systems-manager-parameters.html
Can I kindly request we can override with custom SSM alias to pick up the latest custom AMI to be used with Launch Template?
Regards, Ramesh M
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
I would like Karpenter to be able to use Custom AMI from SSM as described here -- https://docs.aws.amazon.com/autoscaling/ec2/userguide/using-systems-manager-parameters.html
Are you currently working around this issue?
As this feature is not available, we are not able to work around this issue.
Additional Context
No response
Attachments
https://docs.aws.amazon.com/autoscaling/ec2/userguide/using-systems-manager-parameters.html
Community Note