search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.83k stars 9.19k forks source link

[Bug]: Unable to upgrade EKS to 1.30 - unsupported Kubernetes version #37722

Closed rubroboletus closed 5 months ago

rubroboletus commented 5 months ago

Terraform Core Version

1.6.5

AWS Provider Version

5.51.1

Affected Resource(s)

Expected Behavior

upgrade EKS cluster version

Actual Behavior

error is issued, no upgrade of EKS cluster done

Relevant Error/Panic Output Snippet

Error: updating EKS Cluster (eks-dev) version: operation error EKS: UpdateClusterVersion, https response error StatusCode: 400, RequestID: 278c3263-03a3-460e-be58-c6225ebb5e84, InvalidParameterException: unsupported Kubernetes version

  with aws_eks_cluster.cluster,
  on cluster.tf line 101, in resource "aws_eks_cluster" "cluster":
 101: resource "aws_eks_cluster" "cluster" {

Terraform Configuration Files

cluster.tf:

resource "aws_eks_cluster" "cluster" {
  name     = var.cluster_name
  role_arn = aws_iam_role.cluster.arn
  version  = var.cluster_version
  #  enabled_cluster_log_types = ["authenticator", "audit"]

  vpc_config {
    security_group_ids      = [aws_security_group.sg-cluster.id]
    subnet_ids              = concat(data.aws_subnets.priv.ids[*], data.aws_subnets.pub.ids[*])
    endpoint_private_access = "true"
    endpoint_public_access  = "false"
  }

  encryption_config {
    provider {
      key_arn = data.aws_kms_key.envelope.arn
    }
    resources = ["secrets"]
  }

  access_config {
    authentication_mode = "API_AND_CONFIG_MAP"
    bootstrap_cluster_creator_admin_permissions = "false"
  }

  depends_on = [
    aws_iam_role_policy_attachment.cluster-AmazonEKSClusterPolicy,
    aws_iam_role_policy_attachment.cluster-AmazonEKSServicePolicy,
    aws_iam_policy.autoscaler,
    #    aws_cloudwatch_log_group.cluster,
  ]
  lifecycle {
    ignore_changes = [enabled_cluster_log_types]
  }
}

eks-dev.tfvars:

cluster_version = 1.30

Steps to Reproduce

create cluster with version 1.29 try to upgrade to 1.30

Debug Output

aws_eks_cluster.cluster: Modifying... [id=eks-dev]
2024-05-27T12:49:46.118Z [INFO]  Starting apply for aws_eks_cluster.cluster
2024-05-27T12:49:46.119Z [DEBUG] aws_eks_cluster.cluster: applying the planned Update change
2024-05-27T12:49:46.121Z [DEBUG] provider.terraform-provider-aws_v5.51.1_x5: HTTP Request Sent: http.request.header.amz_sdk_request="attempt=1; max=25" http.request.header.amz_sdk_invocation_id=6df13159-546b-420c-aebf-3106e8b18c3e http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************O36U/20240527/eu-central-1/eks/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" tf_aws.signing_region="" http.url=https://eks.eu-central-1.amazonaws.com/clusters/eks-dev/updates tf_req_id=a61eb5a8-4e54-9e05-21f4-a0b71827ece6 tf_rpc=ApplyResourceChange http.request.header.x_amz_security_token="*****" rpc.system=aws-api @module=aws http.method=POST http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.6.5 (+https://www.terraform.io) terraform-provider-aws/5.51.1 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.27.0 os/linux lang/go#1.22.2 md/GOOS#linux md/GOARCH#amd64 api/eks#1.42.4" rpc.method=UpdateClusterVersion tf_aws.sdk=aws-sdk-go-v2 aws.region=eu-central-1 http.request.header.x_amz_date=20240527T124946Z tf_mux_provider="*schema.GRPCProviderServer" tf_resource_type=aws_eks_cluster @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.53/logging/tf_logger.go:45 http.request.header.content_type=application/json http.request_content_length=77 net.peer.name=eks.eu-central-1.amazonaws.com rpc.service=EKS tf_provider_addr=registry.terraform.io/hashicorp/aws
  http.request.body=
  | {"clientRequestToken":"054a7b3f-66c2-4b95-9d2b-59c9a1cbcfd2","version":"1.3"}
   timestamp=2024-05-27T12:49:46.121Z
2024-05-27T12:49:46.386Z [DEBUG] provider.terraform-provider-aws_v5.51.1_x5: HTTP Response Received: http.response.header.access_control_allow_headers="*,Authorization,Date,X-Amz-Date,X-Amz-Security-Token,X-Amz-Target,content-type,x-amz-content-sha256,x-amz-user-agent,x-amzn-platform-id,x-amzn-trace-id" http.status_code=400 tf_aws.sdk=aws-sdk-go-v2 http.response.header.access_control_allow_origin="*" tf_resource_type=aws_eks_cluster @module=aws http.duration=264 http.response.header.x_amzn_errortype=InvalidParameterException tf_rpc=ApplyResourceChange http.response.header.access_control_allow_methods="GET,HEAD,PUT,POST,DELETE,OPTIONS" http.response.header.date="Mon, 27 May 2024 12:49:46 GMT" http.response.header.x_amzn_requestid=7d7b1918-cdd8-49b2-91b0-faa50a3ba8aa http.response.header.x_amzn_trace_id=Root=1-6654816a-75e09ce13ad822b30f85cdc7 rpc.method=UpdateClusterVersion tf_provider_addr=registry.terraform.io/hashicorp/aws aws.region=eu-central-1
  http.response.body=
  | {"clusterName":"eks-dev","nodegroupName":null,"fargateProfileName":null,"addonName":null,"subscriptionId":null,"message":"unsupported Kubernetes version"}
   http.response.header.access_control_expose_headers="x-amzn-errortype,x-amzn-errormessage,x-amzn-trace-id,x-amzn-requestid,x-amz-apigw-id,date" http.response.header.content_type=application/json tf_aws.signing_region="" tf_mux_provider="*schema.GRPCProviderServer" @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.53/logging/tf_logger.go:45 http.response.header.x_amz_apigw_id=YbkopEVzliAEUwQ= http.response_content_length=154 rpc.service=EKS tf_req_id=a61eb5a8-4e54-9e05-21f4-a0b71827ece6 rpc.system=aws-api timestamp=2024-05-27T12:49:46.386Z
2024-05-27T12:49:46.386Z [DEBUG] provider.terraform-provider-aws_v5.51.1_x5: request failed with unretryable error https response error StatusCode: 400, RequestID: 7d7b1918-cdd8-49b2-91b0-faa50a3ba8aa, InvalidParameterException: unsupported Kubernetes version: aws.region=eu-central-1 rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 tf_provider_addr=registry.terraform.io/hashicorp/aws @module=aws tf_mux_provider="*schema.GRPCProviderServer" tf_rpc=ApplyResourceChange rpc.method=UpdateClusterVersion @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.53/logging/tf_logger.go:45 rpc.service=EKS tf_req_id=a61eb5a8-4e54-9e05-21f4-a0b71827ece6 tf_resource_type=aws_eks_cluster timestamp=2024-05-27T12:49:46.386Z
2024-05-27T12:49:46.387Z [ERROR] provider.terraform-provider-aws_v5.51.1_x5: Response contains error diagnostic: tf_resource_type=aws_eks_cluster tf_rpc=ApplyResourceChange diagnostic_summary="updating EKS Cluster (eks-dev) version: operation error EKS: UpdateClusterVersion, https response error StatusCode: 400, RequestID: 7d7b1918-cdd8-49b2-91b0-faa50a3ba8aa, InvalidParameterException: unsupported Kubernetes version" tf_req_id=a61eb5a8-4e54-9e05-21f4-a0b71827ece6 tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/terraform-plugin-go@v0.23.0/tfprotov5/internal/diag/diagnostics.go:58 diagnostic_detail="" tf_proto_version=5.6 @module=sdk.proto diagnostic_severity=ERROR timestamp=2024-05-27T12:49:46.387Z
2024-05-27T12:49:46.390Z [DEBUG] State storage *remote.State declined to persist a state snapshot
2024-05-27T12:49:46.390Z [ERROR] vertex "aws_eks_cluster.cluster" error: updating EKS Cluster (eks-dev) version: operation error EKS: UpdateClusterVersion, https response error StatusCode: 400, RequestID: 7d7b1918-cdd8-49b2-91b0-faa50a3ba8aa, InvalidParameterException: unsupported Kubernetes version
2024-05-27T12:49:46.392Z [DEBUG] states/remote: state read serial is: 218; serial is: 218
2024-05-27T12:49:46.392Z [DEBUG] states/remote: state read lineage is: a791899a-22e7-ecc2-4c1a-06fafd3e87b2; lineage is: a791899a-22e7-ecc2-4c1a-06fafd3e87b2

Error: updating EKS Cluster (eks-dev) version: operation error EKS: UpdateClusterVersion, https response error StatusCode: 400, RequestID: 7d7b1918-cdd8-49b2-91b0-faa50a3ba8aa, InvalidParameterException: unsupported Kubernetes version

  with aws_eks_cluster.cluster,
  on cluster.tf line 101, in resource "aws_eks_cluster" "cluster":
 101: resource "aws_eks_cluster" "cluster" {

2024-05-27T12:49:46.541Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-05-27T12:49:46.541Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-05-27T12:49:46.541Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-05-27T12:49:46.542Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/local/2.4.1/linux_amd64/terraform-provider-local_v2.4.1_x5 pid=864847
2024-05-27T12:49:46.542Z [DEBUG] provider: plugin exited
2024-05-27T12:49:46.542Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/tls/4.0.5/linux_amd64/terraform-provider-tls_v4.0.5_x5 pid=864863
2024-05-27T12:49:46.542Z [DEBUG] provider: plugin exited
2024-05-27T12:49:46.542Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/linux_amd64/terraform-provider-template_v2.2.0_x4 pid=864838
2024-05-27T12:49:46.542Z [DEBUG] provider: plugin exited
2024-05-27T12:49:46.548Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.51.1/linux_amd64/terraform-provider-aws_v5.51.1_x5 pid=864829
2024-05-27T12:49:46.548Z [DEBUG] provider: plugin exited

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

No

github-actions[bot] commented 5 months ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

rubroboletus commented 5 months ago

Same error with terraform version 1.8.4

tanpsingh commented 5 months ago

Same error with terraform eks module version 19.19.0 $ terraform --version Terraform v1.8.4

mifagar commented 5 months ago

Same error with all terraform versions (1.6.5, 1.7.0, 1.8.3, 1.8.4...)

ph-l commented 5 months ago

@rubroboletus Does it work if you add quotes around the value in your tfvars file?

cluster_version = "1.30"

From your logs, it appears that the version was interpreted as a number

2024-05-27T12:49:46.121Z [DEBUG] provider.terraform-provider-aws_v5.51.1_x5: HTTP Request Sent: http.request.header.amz_sdk_request="attempt=1; max=25" http.request.header.amz_sdk_invocation_id=6df13159-546b-420c-aebf-3106e8b18c3e http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************O36U/20240527/eu-central-1/eks/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" tf_aws.signing_region="" http.url=https://eks.eu-central-1.amazonaws.com/clusters/eks-dev/updates tf_req_id=a61eb5a8-4e54-9e05-21f4-a0b71827ece6 tf_rpc=ApplyResourceChange http.request.header.x_amz_security_token="*****" rpc.system=aws-api @module=aws http.method=POST http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.6.5 (+https://www.terraform.io) terraform-provider-aws/5.51.1 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.27.0 os/linux lang/go#1.22.2 md/GOOS#linux md/GOARCH#amd64 api/eks#1.42.4" rpc.method=UpdateClusterVersion tf_aws.sdk=aws-sdk-go-v2 aws.region=eu-central-1 http.request.header.x_amz_date=20240527T124946Z tf_mux_provider="*schema.GRPCProviderServer" tf_resource_type=aws_eks_cluster @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.53/logging/tf_logger.go:45 http.request.header.content_type=application/json http.request_content_length=77 net.peer.name=eks.eu-central-1.amazonaws.com rpc.service=EKS tf_provider_addr=registry.terraform.io/hashicorp/aws
  http.request.body=
  | {"clientRequestToken":"054a7b3f-66c2-4b95-9d2b-59c9a1cbcfd2","version":"1.3"}
   timestamp=2024-05-27T12:49:46.121Z
rubroboletus commented 5 months ago

@ph-l Thank you, this works. Until now, we had it as a number, without quotes, but versions was without "0" at the end.

github-actions[bot] commented 5 months ago

[!WARNING] This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

github-actions[bot] commented 4 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.