tim-finnigan
commented
3 months ago Thanks for reaching out. The modify-instance-metadata-options command involves a call to the ModifyInstanceMetadataOptions API, and that parameter validation is occurring on the API side. We can reach out to the EC2 team about their pattern validation and see if that needs to be updated.
First could you share your debug logs (with any sensitive info redacted) by adding --debug to the command? That should help give more insight into what's going on here. Also, I couldn't find any other reports of this specific issue, so if you're able to provide any additional info regarding the use of EKS here then that may help as well.
kglavin
Describe the bug
With Amazon EKS, the EKS node-group scaler automatically adds some kubernetes tags to the metadata, we then attempt to get ready to modify some other metadata on this instance (separate from the k8s meta data) and aws ec2 modify-instance-metadata-options fails complaining about the k8s meta data being incorrectly formatted.
Expected Behavior
aws ec2 modify-instance-metadata-options should succeed and not be validating some (aws internally inserted metadata).
Current Behavior
WIthin EKS: when the instance is provisioned in the eks cluster using node-group + launch template, the eks cluster node-group (ASG) automatically puts a couple of k8s.io system tags which is conflicting with tag access enable setting.
k8s.io/cluster-autoscaler/enabled
Within AWS EC2 CLI, we then try and change the running instance to allow our automation to add additional metadata and it fails as follows:
aws ec2 modify-instance-metadata-options \ --instance-id i-09659bd7f7501e575 \ --instance-metadata-tags enabled --region us-east-2 returning --->
An error occurred (InvalidParameterValue) when calling the ModifyInstanceMetadataOptions operation: 'k8s.io/cluster-autoscaler/enabled' is not a valid tag key. Tag keys must match pattern ([0-9a-zA-Z\-_+=,.@:]{1,255}), and must not be a reserved name ('.', '..', '_index')
This tag was added "behind the scenes by aws eks orchestration" so either the ec2 cli needs to allow it, or the can the cli team pass along something to the eks team saying they are using an illegal tag that is breaking the cli
Reproduction Steps
WIthin EKS: when the instance is provisioned in the eks cluster using node-group + launch template, the eks cluster node-group (ASG) automatically puts a couple of k8s.io system tags which is conflicting with tag access enable setting.
k8s.io/cluster-autoscaler/enabled
Within AWS EC2 CLI, we then try and change the running instance to allow our automation to add additional metadata and it fails as follows:
aws ec2 modify-instance-metadata-options \ --instance-id i-09659bd7f7501e575 \ --instance-metadata-tags enabled --region us-east-2 returning --->
An error occurred (InvalidParameterValue) when calling the ModifyInstanceMetadataOptions operation: 'k8s.io/cluster-autoscaler/enabled' is not a valid tag key. Tag keys must match pattern ([0-9a-zA-Z\-_+=,.@:]{1,255}), and must not be a reserved name ('.', '..', '_index')
This tag was added "behind the scenes by aws eks orchestration" so either the ec2 cli needs to allow it, or the can the cli team pass along something to the eks team saying they are using an illegal tag that is breaking the cli
Possible Solution
No response
Additional Information/Context
No response
CLI version used
recent
Environment details (OS name and version, etc.)
amazon linux