get the workerpool with the example code and verify the autoscaleEnabled==false
try to resize the workerpool and verify that terraform tries to do it
enable autoscaling for the workerpool
get the workerpool again and verify the autoscaleEnabled==true
try to resize the workerpool and verify that terraform ignores it
logs from VPC test:
% terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
+ create
<= read (data resources)
Terraform will perform the following actions:
# data.ibm_container_cluster_config.cluster_config will be read during apply
# (config refers to values not yet known)
<= data "ibm_container_cluster_config" "cluster_config" {
...
# ibm_container_addons.addons will be created
+ resource "ibm_container_addons" "addons" {
+ cluster = "zoza-tf-test-cluster-12"
+ id = (known after apply)
+ resource_group_id = (known after apply)
+ addons {
...
+ name = "cluster-autoscaler"
+ supported_kube_range = (known after apply)
+ target_version = (known after apply)
+ version = "1.0.7"
+ vlan_spanning_required = (known after apply)
}
}
# ibm_container_vpc_cluster.cluster will be created
+ resource "ibm_container_vpc_cluster" "cluster" {
...
+ disable_public_service_endpoint = false
+ flavor = "bx2.4x16"
+ force_delete_storage = false
+ id = (known after apply)
...
+ name = "zoza-tf-test-cluster-12"
...
+ wait_for_worker_update = true
+ wait_till = "normal"
+ worker_count = 1
+ worker_labels = (known after apply)
+ zones {
...
}
}
# ibm_container_vpc_worker_pool.workerpool will be created
+ resource "ibm_container_vpc_worker_pool" "workerpool" {
+ autoscale_enabled = (known after apply)
+ cluster = (known after apply)
+ flavor = "bx2.4x16"
...
+ worker_count = 1
+ worker_pool_id = (known after apply)
+ worker_pool_name = "workerpool1"
+ zones {
...
}
}
# null_resource.oc_patch will be created
+ resource "null_resource" "oc_patch" {
+ id = (known after apply)
+ triggers = {
+ "autoscaler_enabled" = "false"
}
}
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
ibm_container_vpc_cluster.cluster: Creating...
...
ibm_container_vpc_cluster.cluster: Creation complete after 17m41s [id=<clusterid>]
data.ibm_container_cluster_config.cluster_config: Reading...
ibm_container_addons.addons: Creating...
ibm_container_vpc_worker_pool.workerpool: Creating...
data.ibm_container_cluster_config.cluster_config: Read complete after 5s [id=<clusterid>]
ibm_container_addons.addons: Still creating... [10s elapsed]
ibm_container_vpc_worker_pool.workerpool: Still creating... [10s elapsed]
...
ibm_container_vpc_worker_pool.workerpool: Creation complete after 7m20s [id=<clusterid>/<workerpoolid>]
...
ibm_container_addons.addons: Creation complete after 11m15s [id=zoza-tf-test-cluster-12]
null_resource.oc_patch: Creating...
null_resource.oc_patch: Provisioning with 'local-exec'...
null_resource.oc_patch (local-exec): Executing: ["/bin/sh" "-c" " kubectl patch configmap iks-ca-configmap -n kube-system \\\n -p '{\"data\":{\"workerPoolsConfig.json\":\"[{\\\"name\\\": \\\"default\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":false},{\\\"name\\\": \\\"workerpool1\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":false}]\"}}' --type=merge \\\n --kubeconfig /Users/zoza/42c95bd81837ec0671c57a2a188a8cb255a97c0a322028607af471a39f518c07_<clusterid>_k8sconfig/config.yml\n"]
null_resource.oc_patch (local-exec): configmap/iks-ca-configmap patched
null_resource.oc_patch: Creation complete after 5s [id=578803528917411498]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
------------------
% terraform apply -var worker_count=2
ibm_container_vpc_cluster.cluster: Refreshing state... [id=<clusterid>]
ibm_container_vpc_worker_pool.workerpool: Refreshing state... [id=<clusterid>/<workerpoolid>]
ibm_container_addons.addons: Refreshing state... [id=zoza-tf-test-cluster-12]
null_resource.oc_patch: Refreshing state... [id=578803528917411498]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# ibm_container_vpc_cluster.cluster has been changed
~ resource "ibm_container_vpc_cluster" "cluster" {
...
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using
ignore_changes, the following plan may include actions to undo or respond to these changes.
───────────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# ibm_container_addons.addons will be updated in-place
~ resource "ibm_container_addons" "addons" {
...
# ibm_container_vpc_worker_pool.workerpool will be updated in-place
~ resource "ibm_container_vpc_worker_pool" "workerpool" {
id = "<clusterid>/<workerpoolid>"
~ worker_count = 1 -> 2
# (10 unchanged attributes hidden)
# (1 unchanged block hidden)
}
Plan: 0 to add, 2 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: no
------------------
bluemix-go/examples/container/V2containers/GetWorkerpoolV2 % go run ./main.go -cluster <clusterid> -workerpool <workerpoolid>
{"id":"<workerpoolid>","poolName":"workerpool1","flavor":"bx2.4x16","labels":{"ibm-cloud.kubernetes.io/worker-pool-id":"<workerpoolid>"},"taints":null,"workerCount":1,"isolation":"public","provider":"vpc-gen2","isBalanced":true,"autoscaleEnabled":false,"openshiftLicense":"","lifecycle":{"desiredState":"active","actualState":""},"operatingSystem":"UBUNTU_20_64","zones":[{"id":"au-syd-1","workerCount":1,"autobalanceEnabled":true,"messages":null,"subnets":[{"id":"<subnetid>","primary":true}]}],"vpcID":"<vpcid>"}
------------------
% terraform apply -var enable_clusterautoscaler=true
ibm_container_vpc_cluster.cluster: Refreshing state... [id=<clusterid>]
ibm_container_vpc_worker_pool.workerpool: Refreshing state... [id=<clusterid>/<workerpoolid>]
ibm_container_addons.addons: Refreshing state... [id=zoza-tf-test-cluster-12]
null_resource.oc_patch: Refreshing state... [id=578803528917411498]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
...
# null_resource.oc_patch must be replaced
-/+ resource "null_resource" "oc_patch" {
~ id = "578803528917411498" -> (known after apply)
~ triggers = { # forces replacement
~ "autoscaler_enabled" = "false" -> "true"
}
}
Plan: 1 to add, 1 to change, 1 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
null_resource.oc_patch: Destroying... [id=578803528917411498]
null_resource.oc_patch: Destruction complete after 0s
ibm_container_addons.addons: Modifying... [id=zoza-tf-test-cluster-12]
ibm_container_addons.addons: Still modifying... [id=zoza-tf-test-cluster-12, 10s elapsed]
ibm_container_addons.addons: Modifications complete after 15s [id=zoza-tf-test-cluster-12]
null_resource.oc_patch: Creating...
null_resource.oc_patch: Provisioning with 'local-exec'...
null_resource.oc_patch (local-exec): Executing: ["/bin/sh" "-c" " kubectl patch configmap iks-ca-configmap -n kube-system \\\n -p '{\"data\":{\"workerPoolsConfig.json\":\"[{\\\"name\\\": \\\"default\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":false},{\\\"name\\\": \\\"workerpool1\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":true}]\"}}' --type=merge \\\n --kubeconfig /Users/zoza/42c95bd81837ec0671c57a2a188a8cb255a97c0a322028607af471a39f518c07_<clusterid>_k8sconfig/config.yml\n"]
null_resource.oc_patch (local-exec): configmap/iks-ca-configmap patched
null_resource.oc_patch: Creation complete after 2s [id=838994000405507551]
Apply complete! Resources: 1 added, 1 changed, 1 destroyed.
------------------
% terraform apply -var enable_clusterautoscaler=true -var worker_count=6
ibm_container_vpc_cluster.cluster: Refreshing state... [id=<clusterid>]
ibm_container_vpc_worker_pool.workerpool: Refreshing state... [id=<clusterid>/<workerpoolid>]
ibm_container_addons.addons: Refreshing state... [id=zoza-tf-test-cluster-12]
null_resource.oc_patch: Refreshing state... [id=838994000405507551]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# ibm_container_vpc_worker_pool.workerpool has been changed
~ resource "ibm_container_vpc_worker_pool" "workerpool" {
~ autoscale_enabled = false -> true
id = "<clusterid>/<workerpoolid>"
# (10 unchanged attributes hidden)
# (1 unchanged block hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using
ignore_changes, the following plan may include actions to undo or respond to these changes.
───────────────────────────────────────────────────────────────────────────────────────────────────────────
No changes. Your infrastructure matches the configuration.
Your configuration already matches the changes detected above. If you'd like to update the Terraform state
to match, create and apply a refresh-only plan:
terraform apply -refresh-only
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
---------------
bluemix-go/examples/container/V2containers/GetWorkerpoolV2 % go run ./main.go -cluster <clusterid> -workerpool <workerpoolid>
{"id":"<workerpoolid>","poolName":"workerpool1","flavor":"bx2.4x16","labels":{"ibm-cloud.kubernetes.io/worker-pool-id":"<workerpoolid>"},"taints":null,"workerCount":1,"isolation":"public","provider":"vpc-gen2","isBalanced":true,"autoscaleEnabled":true,"openshiftLicense":"","lifecycle":{"desiredState":"active","actualState":""},"operatingSystem":"UBUNTU_20_64","zones":[{"id":"au-syd-1","workerCount":1,"autobalanceEnabled":true,"messages":null,"subnets":[{"id":"<subnetid>","primary":true}]}],"vpcID":"<vpcid>"}
classic logs:
% terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
+ create
<= read (data resources)
Terraform will perform the following actions:
# data.ibm_container_cluster_config.cluster_config will be read during apply
# (config refers to values not yet known)
<= data "ibm_container_cluster_config" "cluster_config" {
+ admin_certificate = (sensitive value)
+ admin_key = (sensitive value)
+ ca_certificate = (sensitive value)
...
+ token = (sensitive value)
}
# ibm_container_addons.addons will be created
+ resource "ibm_container_addons" "addons" {
+ cluster = "tf-test-cluster-zoza"
+ id = (known after apply)
+ resource_group_id = (known after apply)
+ addons {
...
+ name = "cluster-autoscaler"
+ supported_kube_range = (known after apply)
+ target_version = (known after apply)
+ version = "1.0.7"
+ vlan_spanning_required = (known after apply)
}
}
# ibm_container_cluster.cluster will be created
+ resource "ibm_container_cluster" "cluster" {
...
+ machine_type = "m3c.4x32"
+ name = "tf-test-cluster-zoza"
...
+ wait_for_worker_update = true
+ wait_till = "normal"
+ worker_num = 0
...
}
# ibm_container_worker_pool.workerpool will be created
+ resource "ibm_container_worker_pool" "workerpool" {
+ autoscale_enabled = (known after apply)
...
+ machine_type = "m3c.4x32"
...
+ size_per_zone = 1
+ state = (known after apply)
+ worker_pool_id = (known after apply)
+ worker_pool_name = "ubuntu18pool"
+ zones = (known after apply)
}
# ibm_container_worker_pool_zone_attachment.zones will be created
+ resource "ibm_container_worker_pool_zone_attachment" "zones" {
...
}
# null_resource.oc_patch will be created
+ resource "null_resource" "oc_patch" {
+ id = (known after apply)
+ triggers = {
+ "autoscaler_enabled" = "false"
}
}
Plan: 5 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
ibm_container_cluster.cluster: Creating...
...
ibm_container_cluster.cluster: Creation complete after 19m28s [id=<clusterid>]
data.ibm_container_cluster_config.cluster_config: Reading...
ibm_container_addons.addons: Creating...
ibm_container_worker_pool.workerpool: Creating...
ibm_container_worker_pool.workerpool: Creation complete after 3s [id=<clusterid>/<workerpoolid>]
ibm_container_worker_pool_zone_attachment.zones: Creating...
data.ibm_container_cluster_config.cluster_config: Read complete after 6s [id=<clusterid>]
ibm_container_addons.addons: Still creating... [10s elapsed]
ibm_container_worker_pool_zone_attachment.zones: Still creating... [10s elapsed]
...
ibm_container_addons.addons: Creation complete after 7m41s [id=tf-test-cluster-zoza]
null_resource.oc_patch: Creating...
null_resource.oc_patch: Provisioning with 'local-exec'...
null_resource.oc_patch (local-exec): Executing: ["/bin/sh" "-c" " kubectl patch configmap iks-ca-configmap -n kube-system \\\n -p '{\"data\":{\"workerPoolsConfig.json\":\"[{\\\"name\\\": \\\"default\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":false},{\\\"name\\\": \\\"<workerpoolid>\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":false}]\"}}' --type=merge \\\n --kubeconfig /Users/zoza/ee2506f845f94cf835354947613fe7c3001bef2c3618900d9a930e208a3d32af_<clusterid>_k8sconfig/config.yml\n"]
ibm_container_worker_pool_zone_attachment.zones: Still creating... [7m40s elapsed]
null_resource.oc_patch (local-exec): configmap/iks-ca-configmap patched
null_resource.oc_patch: Creation complete after 4s [id=1137090567378279591]
...
ibm_container_worker_pool_zone_attachment.zones: Creation complete after 8m30s [id=<clusterid>/<workerpoolid>/syd01]
Apply complete! Resources: 5 added, 0 changed, 0 destroyed.
-----------------------------
% terraform apply -var worker_count=2
ibm_container_cluster.cluster: Refreshing state... [id=<clusterid>]
ibm_container_worker_pool.workerpool: Refreshing state... [id=<clusterid>/<workerpoolid>]
ibm_container_addons.addons: Refreshing state... [id=tf-test-cluster-zoza]
ibm_container_worker_pool_zone_attachment.zones: Refreshing state... [id=<clusterid>/<workerpoolid>/syd01]
null_resource.oc_patch: Refreshing state... [id=1137090567378279591]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# ibm_container_cluster.cluster has been changed
~ resource "ibm_container_cluster" "cluster" {
...
}
# ibm_container_worker_pool.workerpool has been changed
~ resource "ibm_container_worker_pool" "workerpool" {
...
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using
ignore_changes, the following plan may include actions to undo or respond to these changes.
───────────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# ibm_container_worker_pool.workerpool will be updated in-place
~ resource "ibm_container_worker_pool" "workerpool" {
id = "<clusterid>/<workerpoolid>"
~ size_per_zone = 1 -> 2
# (12 unchanged attributes hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: no
----------------------
bluemix-go/examples/container/getWorkerPool % go run ./main.go -resourcegroup <resourcegroupid> -region au-syd -cluster <clusterid> -workerpool <workerpoolid>
{"name":"ubuntu18pool","sizePerZone":1,"machineType":"m3c.4x32.encrypted","isolation":"public","labels":{"ibm-cloud.kubernetes.io/worker-pool-id":"<workerpoolid>"},"id":"<workerpoolid>","region":"au-syd","state":"active","reasonForDelete":"","operatingSystem":"UBUNTU_20_64","isBalanced":true,"autoscaleEnabled":false,"zones":[{"privateVlan":"<privatevlanid>","publicVlan":"<publicvlanid>","id":"syd01","workerCount":1}]}
---------------------
% terraform apply -var enable_clusterautoscaler=true
ibm_container_cluster.cluster: Refreshing state... [id=<clusterid>]
ibm_container_worker_pool.workerpool: Refreshing state... [id=<clusterid>/<workerpoolid>]
ibm_container_addons.addons: Refreshing state... [id=tf-test-cluster-zoza]
ibm_container_worker_pool_zone_attachment.zones: Refreshing state... [id=<clusterid>/<workerpoolid>/syd01]
null_resource.oc_patch: Refreshing state... [id=1137090567378279591]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
....
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using
ignore_changes, the following plan may include actions to undo or respond to these changes.
───────────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
-/+ destroy and then create replacement
Terraform will perform the following actions:
# null_resource.oc_patch must be replaced
-/+ resource "null_resource" "oc_patch" {
~ id = "1137090567378279591" -> (known after apply)
~ triggers = { # forces replacement
~ "autoscaler_enabled" = "false" -> "true"
}
}
Plan: 1 to add, 0 to change, 1 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
null_resource.oc_patch: Destroying... [id=1137090567378279591]
null_resource.oc_patch: Destruction complete after 0s
null_resource.oc_patch: Creating...
null_resource.oc_patch: Provisioning with 'local-exec'...
null_resource.oc_patch (local-exec): Executing: ["/bin/sh" "-c" " kubectl patch configmap iks-ca-configmap -n kube-system \\\n -p '{\"data\":{\"workerPoolsConfig.json\":\"[{\\\"name\\\": \\\"default\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":false},{\\\"name\\\": \\\"<workerpoolid>\\\",\\\"minSize\\\": 1,\\\"maxSize\\\": 2,\\\"enabled\\\":true}]\"}}' --type=merge \\\n --kubeconfig /Users/zoza/ee2506f845f94cf835354947613fe7c3001bef2c3618900d9a930e208a3d32af_<clusterid>_k8sconfig/config.yml\n"]
null_resource.oc_patch (local-exec): configmap/iks-ca-configmap patched
null_resource.oc_patch: Creation complete after 2s [id=8667523543605951666]
---------------------
% terraform apply -var enable_clusterautoscaler=true -var worker_count=6
ibm_container_cluster.cluster: Refreshing state... [id=<clusterid>]
ibm_container_worker_pool.workerpool: Refreshing state... [id=<clusterid>/<workerpoolid>]
ibm_container_addons.addons: Refreshing state... [id=tf-test-cluster-zoza]
ibm_container_worker_pool_zone_attachment.zones: Refreshing state... [id=<clusterid>/<workerpoolid>/syd01]
null_resource.oc_patch: Refreshing state... [id=8667523543605951666]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# ibm_container_worker_pool_zone_attachment.zones has been changed
~ resource "ibm_container_worker_pool_zone_attachment" "zones" {
id = "<clusterid>/<workerpoolid>/syd01"
~ worker_count = 1 -> 2
# (7 unchanged attributes hidden)
}
# ibm_container_cluster.cluster has been changed
~ resource "ibm_container_cluster" "cluster" {
...
}
# ibm_container_worker_pool.workerpool has been changed
~ resource "ibm_container_worker_pool" "workerpool" {
~ autoscale_enabled = false -> true
id = "<clusterid>/<workerpoolid>"
~ zones = [
~ {
~ worker_count = 1 -> 2
# (3 unchanged elements hidden)
},
]
# (11 unchanged attributes hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using
ignore_changes, the following plan may include actions to undo or respond to these changes.
───────────────────────────────────────────────────────────────────────────────────────────────────────────
No changes. Your infrastructure matches the configuration.
Your configuration already matches the changes detected above. If you'd like to update the Terraform state
to match, create and apply a refresh-only plan:
terraform apply -refresh-only
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
---------------------
bluemix-go/examples/container/getWorkerPool % go run ./main.go -resourcegroup <resourcegroupid> -region au-syd -cluster <clusterid> -workerpool <workerpoolid>
{"name":"ubuntu18pool","sizePerZone":1,"machineType":"m3c.4x32.encrypted","isolation":"public","labels":{"ibm-cloud.kubernetes.io/worker-pool-id":"<workerpoolid>"},"id":"<workerpoolid>","region":"au-syd","state":"active","reasonForDelete":"","operatingSystem":"UBUNTU_20_64","isBalanced":true,"autoscaleEnabled":true,"zones":[{"privateVlan":"<privatevlanid>","publicVlan":"<publicvlanid>","id":"syd01","workerCount":2}]}
I've tested with the following scenario:
logs from VPC test:
classic logs: