kubernetes-sigs / karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Apache License 2.0
552 stars 184 forks source link

Upgarde issue from 35 to 1.0 #1713

Open sree42111 opened 2 hours ago

sree42111 commented 2 hours ago

Description

We have updated Karpenter from version 0.35 to 0.36, and then to 0.37, v1.0. We have applied the patches according to the documentation. However, we are currently encountering the following error, despite having the correct EC2 create tag permission.

UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::5482:assumed-role/KarpenterNodeRole-DR-R4E-N/17272406301 is not authorized to perform: ec2:CreateTags on resource: arn:aws:ec2:us-east-1:5480:spot-instances-request/* because no identity-based policy allows the ec2:CreateTags action

aws sts decode-authorization-message { "DecodedMessage": "{\"allowed\":false,\"explicitDeny\":false,\"matchedStatements\":{\"items\":[]},\"failures\":{\"items\":[]},\"context\":{\"principal\":{\"id\":\"AROAX7JP:1727241\",\"arn\":\"arn:aws:sts::548:assumed-role/KarpenterNodeRole-DR-P-N/1727223001\"},\"action\":\"RunInstances\",\"resource\":\"arn:aws:ec2:us-east-1:54820:spot-instances-request/\",\"conditions\":{\"items\":[{\"key\":\"aws:Region\",\"values\":{\"items\":[{\"value\":\"us-east-1\"}]}},{\"key\":\"aws:ID\",\"values\":{\"items\":[{\"value\":\"\"}]}},{\"key\":\"aws:Service\",\"values\":{\"items\":[{\"value\":\"ec2\"}]}},{\"key\":\"aws:Resource\",\"values\":{\"items\":[{\"value\":\"spot-instances-request/\"}]}},{\"key\":\"aws:Type\",\"values\":{\"items\":[{\"value\":\"spot-instances-request\"}]}},{\"key\":\"aws:Account\",\"values\":{\"items\":[{\"value\":\"54\"}]}},{\"key\":\"ec2:Region\",\"values\":{\"items\":[{\"value\":\"us-east-1\"}]}},{\"key\":\"aws:ARN\",\"values\":{\"items\":[{\"value\":\"arn:aws:ec2:us-east-1:548:spot-instances-request/\"}]}}]}}}" } Karpneter policy :

{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowScopedEC2InstanceActions", "Effect": "Allow", "Resource": [ "arn:aws:ec2:us-east-1::image/", "arn:aws:ec2:us-east-1::snapshot/", "arn:aws:ec2:us-east-1::spot-instances-request/", "arn:aws:ec2:us-east-1::security-group/", "arn:aws:ec2:us-east-1::subnet/", "arn:aws:ec2:us-east-1::launch-template/" ], "Action": [ "ec2:RunInstances", "ec2:CreateFleet" ] }, { "Sid": "AllowScopedEC2LaunchTemplateActions", "Effect": "Allow", "Resource": "arn:aws:ec2:us-east-1::launch-template/", "Action": "ec2:CreateLaunchTemplate", "Condition": { "StringEquals": { "aws:RequestTag/kubernetes.io/cluster/DR-P-N": "owned" }, "StringLike": { "aws:RequestTag/karpenter.sh/provisioner-name": "" } } }, { "Sid": "AllowScopedEC2InstanceActionsWithTags", "Effect": "Allow", "Resource": [ "arn:aws:ec2:us-east-1::fleet/", "arn:aws:ec2:us-east-1::instance/", "arn:aws:ec2:us-east-1::volume/", "arn:aws:ec2:us-east-1::network-interface/" ], "Action": [ "ec2:RunInstances", "ec2:CreateFleet" ], "Condition": { "StringEquals": { "aws:RequestTag/kubernetes.io/cluster/DR-P-N": "owned" }, "StringLike": { "aws:RequestTag/karpenter.sh/provisioner-name": "" } } }, { "Sid": "AllowScopedResourceCreationTagging", "Effect": "Allow", "Resource": [ "arn:aws:ec2:us-east-1::fleet/", "arn:aws:ec2:us-east-1::instance/", "arn:aws:ec2:us-east-1::volume/", "arn:aws:ec2:us-east-1::network-interface/", "arn:aws:ec2:us-east-1::launch-template/" ], "Action": "ec2:CreateTags", "Condition": { "StringEquals": { "aws:RequestTag/kubernetes.io/cluster/DR-P-R4E-N": "owned", "ec2:CreateAction": [ "RunInstances", "CreateFleet", "CreateLaunchTemplate" ] }, "StringLike": { "aws:RequestTag/karpenter.sh/provisioner-name": "*"

k8s-ci-robot commented 2 hours ago

This issue is currently awaiting triage.

If Karpenter contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.