Cluster CCE Does Not Recreate When Subnet CIDR Changes

[Mobydack]

Terraform and Provider Versions

• Terraform Version: v1.9.2

• Provider Version: sbercloud-terraform/sbercloud v1.11.6

  Affected Resources

• sbercloud_vpc_subnet

• sbercloud_cce_cluster

  Terraform Configuration Files

  Subnet Resource

  resource "sbercloud_vpc_subnet" "k8s_staging_pods" {
  cidr          = "10.0.2.0/22"
  gateway_ip    = "10.0.2.1"
  name          = "k8s-staging-pods"
  primary_dns   = var.internal_dns_master
  secondary_dns = "8.8.8.8"
  vpc_id        = sbercloud_vpc.k8s_staging.id
  }

  CCE Cluster Resource

  resource "sbercloud_cce_cluster" "k8s_staging" {
  name      = "k8s-staging"
  flavor_id = "cce.s2.small"
  vpc_id    = sbercloud_vpc.k8s_staging.id
  subnet_id = sbercloud_vpc_subnet.k8s_staging_nodes.id
  
  masters {
  availability_zone = "ru-moscow-1a"
  }
  
  masters {
  availability_zone = "ru-moscow-1a"
  }
  
  masters {
  availability_zone = "ru-moscow-1c"
  }
  
  cluster_type           = "VirtualMachine"
  cluster_version        = "v1.29"
  container_network_type = "eni"
  kube_proxy_mode        = "ipvs"
  
  eni_subnet_id = join(",", [
  sbercloud_vpc_subnet.k8s_staging_pods.ipv4_subnet_id
  ])
  
  eip = sbercloud_vpc_eip.k8s_staging_eip.address
  
  service_network_cidr = "10.0.6.0/22"
  
  tags = {
  Environment = "Staging"
  Application = "Kubernetes"
  }
  }

  ---

  Problem Description

  Steps to Reproduce

  1. Create the subnet and cluster using the configuration provided above.
  2. Modify the cidr block of the resource sbercloud_vpc_subnet.k8s_staging_pods (e.g., change from 10.0.2.0/22 to 10.0.4.0/22).
  3. Run terraform apply.

Expected Behavior

The sbercloud_cce_cluster resource should detect the dependency on the updated subnet and recreate accordingly. Subnet deletion and recreation should not cause timeouts or conflicts.

Actual Behavior

When the sbercloud_vpc_subnet.k8s_staging_pods CIDR is changed, Terraform attempts to delete the subnet but fails because it is still in use by the CCE cluster.

This results in the following error:

│ Error: error deleting Subnet: timeout while waiting for state to become 'DELETED' (last state: 'ACTIVE', timeout: 10m0s)

---

Planned Resource Changes

    # other resources will be updated (for example, the ingress add-on), but not the CCE cluster
    #sbercloud_vpc_subnet.k8s_staging_pods must be replaced
-/+ resource "sbercloud_vpc_subnet" "k8s_staging_pods" {
      ~ cidr              = "10.0.2.0/22" -> "10.0.4.0/22" # forces replacement
      ~ gateway_ip        = "10.0.2.1" -> "10.0.4.1" # forces replacement
    }

---

Important Notes

• This issue arises because the CCE cluster does not automatically detect and manage the dependency on the subnet when its cidr block changes.
• The cluster locks the subnet, causing a timeout during deletion.
• To solve the problem, we may need to link it to the lifecycle

  lifecycle {
  replace_triggered_by = [
    sbercloud_vpc_subnet.k8s_staging_nodes.cidr,
    sbercloud_vpc_subnet.k8s_staging_pods.cidr,
  ]
  }