couchbasecloud / terraform-provider-couchbase-capella

Terraform Couchbase Capella Provider: Deploy, update, and manage Couchbase Capella infrastructure as code through HashiCorp Terraform
Apache License 2.0
32 stars 4 forks source link

AV-76498 | manage GSI indexes #233

Open l0n3star opened 1 month ago

l0n3star commented 1 month ago

Jira

Description

Feature to manage primary and secondary indexes.

Design doc: https://docs.google.com/document/d/1pTjOIA_DuvEkpj83QQC-9VRYFuOUhUcy-NIOTYh4mVM/edit?usp=sharing

Type of Change

Manual Testing Approach

How was this change tested and do you have evidence? (REQUIRED: Select at least 1)

Testing

Testing // create new indexes ``` terraform apply ╷ │ Warning: Provider development overrides are in effect │ │ The following provider development overrides are set in the CLI configuration: │ - couchbasecloud/couchbase-capella in /Users/$USER/GolandProjects/terraform-provider-couchbase-capella/bin │ │ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. ╵ Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # couchbase-capella_query_indexes.new_indexes["idx1"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field1", ] + index_name = "idx1" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = false + num_partition = 8 + num_replica = 0 } } # couchbase-capella_query_indexes.new_indexes["idx2"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field2", ] + index_name = "idx2" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = false + num_partition = 8 + num_replica = 0 } } # couchbase-capella_query_indexes.new_indexes["idx3"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field3", ] + index_name = "idx3" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = false + num_partition = 8 + num_replica = 0 } } # couchbase-capella_query_indexes.new_indexes["idx4"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field5", ] + index_name = "idx4" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = false + num_partition = 8 + num_replica = 0 } } # couchbase-capella_query_indexes.new_indexes["idx5"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field5", ] + index_name = "idx5" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = false + num_partition = 8 + num_replica = 0 } } Plan: 5 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes couchbase-capella_query_indexes.new_indexes["idx2"]: Creating... couchbase-capella_query_indexes.new_indexes["idx5"]: Creating... couchbase-capella_query_indexes.new_indexes["idx3"]: Creating... couchbase-capella_query_indexes.new_indexes["idx1"]: Creating... couchbase-capella_query_indexes.new_indexes["idx4"]: Creating... couchbase-capella_query_indexes.new_indexes["idx3"]: Creation complete after 4s couchbase-capella_query_indexes.new_indexes["idx2"]: Creation complete after 9s couchbase-capella_query_indexes.new_indexes["idx5"]: Still creating... [10s elapsed] couchbase-capella_query_indexes.new_indexes["idx1"]: Still creating... [10s elapsed] couchbase-capella_query_indexes.new_indexes["idx4"]: Still creating... [10s elapsed] couchbase-capella_query_indexes.new_indexes["idx1"]: Creation complete after 14s couchbase-capella_query_indexes.new_indexes["idx4"]: Creation complete after 19s couchbase-capella_query_indexes.new_indexes["idx5"]: Still creating... [20s elapsed] couchbase-capella_query_indexes.new_indexes["idx5"]: Creation complete after 24s Apply complete! Resources: 5 added, 0 changed, 0 destroyed. ``` // drop indexes ``` terraform apply ╷ │ Warning: Provider development overrides are in effect │ │ The following provider development overrides are set in the CLI configuration: │ - couchbasecloud/couchbase-capella in /Users/$USER/GolandProjects/terraform-provider-couchbase-capella/bin │ │ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. ╵ data.couchbase-capella_query_indexes.list: Reading... couchbase-capella_query_indexes.new_indexes["idx5"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx3"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx4"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx2"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx1"]: Refreshing state... data.couchbase-capella_query_indexes.list: Read complete after 0s Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: - destroy Terraform will perform the following actions: # couchbase-capella_query_indexes.new_indexes["idx1"] will be destroyed # (because couchbase-capella_query_indexes.new_indexes is not in configuration) - resource "couchbase-capella_query_indexes" "new_indexes" { - bucket_name = "test" -> null - cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - collection_name = "test" -> null - index_keys = [ - "field1", ] -> null - index_name = "idx1" -> null - organization_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - project_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - scope_name = "test" -> null - with = { - defer_build = false -> null - num_partition = 8 -> null - num_replica = 0 -> null } -> null } # couchbase-capella_query_indexes.new_indexes["idx2"] will be destroyed # (because couchbase-capella_query_indexes.new_indexes is not in configuration) - resource "couchbase-capella_query_indexes" "new_indexes" { - bucket_name = "test" -> null - cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - collection_name = "test" -> null - index_keys = [ - "field2", ] -> null - index_name = "idx2" -> null - organization_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - project_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - scope_name = "test" -> null - with = { - defer_build = false -> null - num_partition = 8 -> null - num_replica = 0 -> null } -> null } # couchbase-capella_query_indexes.new_indexes["idx3"] will be destroyed # (because couchbase-capella_query_indexes.new_indexes is not in configuration) - resource "couchbase-capella_query_indexes" "new_indexes" { - bucket_name = "test" -> null - cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - collection_name = "test" -> null - index_keys = [ - "field3", ] -> null - index_name = "idx3" -> null - organization_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - project_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - scope_name = "test" -> null - with = { - defer_build = false -> null - num_partition = 8 -> null - num_replica = 0 -> null } -> null } # couchbase-capella_query_indexes.new_indexes["idx4"] will be destroyed # (because couchbase-capella_query_indexes.new_indexes is not in configuration) - resource "couchbase-capella_query_indexes" "new_indexes" { - bucket_name = "test" -> null - cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - collection_name = "test" -> null - index_keys = [ - "field5", ] -> null - index_name = "idx4" -> null - organization_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - project_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - scope_name = "test" -> null - with = { - defer_build = false -> null - num_partition = 8 -> null - num_replica = 0 -> null } -> null } # couchbase-capella_query_indexes.new_indexes["idx5"] will be destroyed # (because couchbase-capella_query_indexes.new_indexes is not in configuration) - resource "couchbase-capella_query_indexes" "new_indexes" { - bucket_name = "test" -> null - cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - collection_name = "test" -> null - index_keys = [ - "field5", ] -> null - index_name = "idx5" -> null - organization_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - project_id = "ffffffff-aaaa-1414-eeee-000000000000" -> null - scope_name = "test" -> null - with = { - defer_build = false -> null - num_partition = 8 -> null - num_replica = 0 -> null } -> null } Plan: 0 to add, 0 to change, 5 to destroy. Changes to Outputs: + list_indexes = { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + data = [ + { + definition = "CREATE INDEX `idx1` ON `test`.`test`.`test`(`field1`)" + index_name = "idx1" }, + { + definition = "CREATE INDEX `idx2` ON `test`.`test`.`test`(`field2`)" + index_name = "idx2" }, + { + definition = "CREATE INDEX `idx3` ON `test`.`test`.`test`(`field3`)" + index_name = "idx3" }, + { + definition = "CREATE INDEX `idx4` ON `test`.`test`.`test`(`field5`)" + index_name = "idx4" }, + { + definition = "CREATE INDEX `idx5` ON `test`.`test`.`test`(`field5`)" + index_name = "idx5" }, ] + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" } Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes couchbase-capella_query_indexes.new_indexes["idx1"]: Destroying... couchbase-capella_query_indexes.new_indexes["idx3"]: Destroying... couchbase-capella_query_indexes.new_indexes["idx5"]: Destroying... couchbase-capella_query_indexes.new_indexes["idx2"]: Destroying... couchbase-capella_query_indexes.new_indexes["idx4"]: Destroying... couchbase-capella_query_indexes.new_indexes["idx5"]: Destruction complete after 0s couchbase-capella_query_indexes.new_indexes["idx3"]: Destruction complete after 0s couchbase-capella_query_indexes.new_indexes["idx2"]: Destruction complete after 0s couchbase-capella_query_indexes.new_indexes["idx1"]: Destruction complete after 0s couchbase-capella_query_indexes.new_indexes["idx4"]: Destruction complete after 0s Apply complete! Resources: 0 added, 0 changed, 5 destroyed. ``` // list indexes ``` terraform plan ╷ │ Warning: Provider development overrides are in effect │ │ The following provider development overrides are set in the CLI configuration: │ - couchbasecloud/couchbase-capella in /Users/$USER/GolandProjects/terraform-provider-couchbase-capella/bin │ │ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. ╵ data.couchbase-capella_query_indexes.list: Reading... couchbase-capella_query_indexes.new_indexes["idx5"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx2"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx3"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx1"]: Refreshing state... couchbase-capella_query_indexes.new_indexes["idx4"]: Refreshing state... data.couchbase-capella_query_indexes.list: Read complete after 1s Changes to Outputs: + list_indexes = { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + data = [ + { + definition = "CREATE INDEX `idx1` ON `test`.`test`.`test`(`field1`)" + index_name = "idx1" }, + { + definition = "CREATE INDEX `idx2` ON `test`.`test`.`test`(`field2`)" + index_name = "idx2" }, + { + definition = "CREATE INDEX `idx3` ON `test`.`test`.`test`(`field3`)" + index_name = "idx3" }, + { + definition = "CREATE INDEX `idx4` ON `test`.`test`.`test`(`field5`)" + index_name = "idx4" }, + { + definition = "CREATE INDEX `idx5` ON `test`.`test`.`test`(`field5`)" + index_name = "idx5" }, ] + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" } You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure. ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now. ``` // import index idx1 in test.test.est ``` couchbase-capella_query_indexes.idx1: Importing from ID "index_name=idx1,collection_name=test,scope_name=test,bucket_name=test,cluster_id=ffffffff-aaaa-1414-eeee-000000000000,project_id=ffffffff-aaaa-1414-eeee-000000000000,organization_id=ffffffff-aaaa-1414-eeee-000000000000"... couchbase-capella_query_indexes.idx1: Import prepared! Prepared couchbase-capella_query_indexes for import couchbase-capella_query_indexes.idx1: Refreshing state... 2024-09-25T12:43:09.513-0700 [WARN] Provider "registry.terraform.io/couchbasecloud/couchbase-capella" produced an unexpected new value for couchbase-capella_query_indexes.idx1 during refresh. - .collection_name: was null, but now cty.StringVal("test") - .index_keys: was null, but now cty.ListVal([]cty.Value{cty.StringVal("`c0`")}) - .index_name: was cty.StringVal("index_name=idx1,collection_name=test,scope_name=test,bucket_name=test,cluster_id=ffffffff-aaaa-1414-eeee-000000000000,project_id=ffffffff-aaaa-1414-eeee-000000000000,organization_id=ffffffff-aaaa-1414-eeee-000000000000"), but now cty.StringVal("idx1") - .is_primary: was null, but now cty.False - .project_id: was null, but now cty.StringVal("ffffffff-aaaa-1414-eeee-000000000000") - .scope_name: was null, but now cty.StringVal("test") - .where: was null, but now cty.StringVal("") - .bucket_name: was null, but now cty.StringVal("test") - .organization_id: was null, but now cty.StringVal("ffffffff-aaaa-1414-eeee-000000000000") - .cluster_id: was null, but now cty.StringVal("ffffffff-aaaa-1414-eeee-000000000000") Import successful! The resources that were imported are shown above. These resources are now in your Terraform state and will henceforth be managed by Terraform. ``` // create and build 5 deferred indexes ``` terraform apply ╷ │ Warning: Provider development overrides are in effect │ │ The following provider development overrides are set in the CLI configuration: │ - couchbasecloud/couchbase-capella in /Users/hiteshwalia/GolandProjects/terraform-provider-couchbase-capella/bin │ │ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published │ releases. ╵ Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create <= read (data resources) Terraform will perform the following actions: # data.couchbase-capella_query_index_monitor.mon_indexes will be read during apply # (depends on a resource or a module with changes pending) <= data "couchbase-capella_query_index_monitor" "mon_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + indexes = [ + "idx1", + "idx2", + "idx3", + "idx4", + "idx5", ] + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" } # couchbase-capella_query_indexes.build_idx will be created + resource "couchbase-capella_query_indexes" "build_idx" { + bucket_name = "test" + build_indexes = [ + "idx1", + "idx2", + "idx3", + "idx4", + "idx5", ] + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" } # couchbase-capella_query_indexes.new_indexes["idx1"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field1", ] + index_name = "idx1" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = true } } # couchbase-capella_query_indexes.new_indexes["idx2"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field2", ] + index_name = "idx2" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = true } } # couchbase-capella_query_indexes.new_indexes["idx3"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field3", ] + index_name = "idx3" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = true } } # couchbase-capella_query_indexes.new_indexes["idx4"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field5", ] + index_name = "idx4" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = true } } # couchbase-capella_query_indexes.new_indexes["idx5"] will be created + resource "couchbase-capella_query_indexes" "new_indexes" { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + index_keys = [ + "field5", ] + index_name = "idx5" + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" + with = { + defer_build = true } } Plan: 6 to add, 0 to change, 0 to destroy. Changes to Outputs: + watch_indexes = { + bucket_name = "test" + cluster_id = "ffffffff-aaaa-1414-eeee-000000000000" + collection_name = "test" + indexes = [ + "idx1", + "idx2", + "idx3", + "idx4", + "idx5", ] + organization_id = "ffffffff-aaaa-1414-eeee-000000000000" + project_id = "ffffffff-aaaa-1414-eeee-000000000000" + scope_name = "test" } Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes couchbase-capella_query_indexes.new_indexes["idx1"]: Creating... couchbase-capella_query_indexes.build_idx: Creating... couchbase-capella_query_indexes.new_indexes["idx2"]: Creating... couchbase-capella_query_indexes.new_indexes["idx5"]: Creating... couchbase-capella_query_indexes.new_indexes["idx3"]: Creating... couchbase-capella_query_indexes.new_indexes["idx4"]: Creating... couchbase-capella_query_indexes.new_indexes["idx1"]: Creation complete after 1s couchbase-capella_query_indexes.new_indexes["idx2"]: Creation complete after 2s couchbase-capella_query_indexes.new_indexes["idx5"]: Creation complete after 3s couchbase-capella_query_indexes.new_indexes["idx4"]: Creation complete after 4s couchbase-capella_query_indexes.new_indexes["idx3"]: Creation complete after 5s couchbase-capella_query_indexes.build_idx: Still creating... [10s elapsed] couchbase-capella_query_indexes.build_idx: Still creating... [20s elapsed] couchbase-capella_query_indexes.build_idx: Still creating... [30s elapsed] couchbase-capella_query_indexes.build_idx: Still creating... [40s elapsed] couchbase-capella_query_indexes.build_idx: Still creating... [50s elapsed] couchbase-capella_query_indexes.build_idx: Still creating... [1m0s elapsed] couchbase-capella_query_indexes.build_idx: Creation complete after 1m1s data.couchbase-capella_query_index_monitor.mon_indexes: Reading... data.couchbase-capella_query_index_monitor.mon_indexes: Still reading... [10s elapsed] data.couchbase-capella_query_index_monitor.mon_indexes: Still reading... [20s elapsed] data.couchbase-capella_query_index_monitor.mon_indexes: Still reading... [30s elapsed] data.couchbase-capella_query_index_monitor.mon_indexes: Still reading... [40s elapsed] data.couchbase-capella_query_index_monitor.mon_indexes: Still reading... [50s elapsed] data.couchbase-capella_query_index_monitor.mon_indexes: Still reading... [1m0s elapsed] data.couchbase-capella_query_index_monitor.mon_indexes: Read complete after 1m1s Apply complete! Resources: 6 added, 0 changed, 0 destroyed. Outputs: watch_indexes = { "bucket_name" = "test" "cluster_id" = "ffffffff-aaaa-1414-eeee-000000000000" "collection_name" = "test" "indexes" = toset([ "idx1", "idx2", "idx3", "idx4", "idx5", ]) "organization_id" = "ffffffff-aaaa-1414-eeee-000000000000" "project_id" = "ffffffff-aaaa-1414-eeee-000000000000" "scope_name" = "test" } ```

Required Checklist:

Further comments

I left I have made sure that no schema field is marked with both requiresReplace and computed unchecked since scope_name should be set to requiresReplace but since it has default value it must be computed