[Bug]: plugin crashed during creation of mongodbatlas_search_index

gengdupuis commented 1 month ago

Is there an existing issue for this?

[X] I have searched the existing issues

Provider Version

v1.15.3

Terraform Version

1.8.2

Terraform Edition

Terraform Open Source (OSS)

Current Behavior

for index creation, the terraform plan specified the indexes would be created, and the terraform plan succeeded. However, when checking in MongoDB Atlas, the indexes were not there. Furthermore, any subsequent terraform plan detected that the same indexes would be created again.

This lead to adding wait_for_index_build_completion = true in the resource to make terraform wait for the index to be active before considering the resource built. The terraform plan worked correctly, but the terraform apply failed with the plugin crashing (logs pasted below).

Other things to note:

the cluster on which this occurs is empty, meaning that there are still no databases or collections existing in it. We would expect those to be created upon the index being created.
this happened on an M20 REPLICASET cluster

Terraform configuration to reproduce the issue

resource "mongodbatlas_search_index" "test_search_index" {
  for_each = toset(["FirstTestCollection", "SecondTestCollection", "ThirdTestCollection"])

  name             = "TestIndex"
  project_id       = "<some-project-id>"
  cluster_name     = "<some-cluster-name>"
  collection_name  = each.key
  database         = "TestDB"
  mappings_dynamic = true

  # adding this made the plugin crash
  wait_for_index_build_completion = true
}

Steps To Reproduce

Create search indexes using the above resource code, replacing <placeholders>
Terraform apply

Logs

Stack trace from the terraform-provider-mongodbatlas_v1.15.3 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x1725730]

goroutine 83 [running]:
github.com/mongodb/terraform-provider-mongodbatlas/internal/service/searchindex.resourceCreate.resourceSearchIndexRefreshFunc.func1()
    github.com/mongodb/terraform-provider-mongodbatlas/internal/service/searchindex/resource_search_index.go:578 +0xb0
github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext.func1()
    github.com/hashicorp/terraform-plugin-sdk/v2@v2.33.0/helper/retry/state.go:113 +0x1b0
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext in goroutine 51
    github.com/hashicorp/terraform-plugin-sdk/v2@v2.33.0/helper/retry/state.go:86 +0x23b
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x1725730]

goroutine 82 [running]:
github.com/mongodb/terraform-provider-mongodbatlas/internal/service/searchindex.resourceCreate.resourceSearchIndexRefreshFunc.func1()
    github.com/mongodb/terraform-provider-mongodbatlas/internal/service/searchindex/resource_search_index.go:578 +0xb0
github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext.func1()
    github.com/hashicorp/terraform-plugin-sdk/v2@v2.33.0/helper/retry/state.go:113 +0x1b0
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext in goroutine 43
    github.com/hashicorp/terraform-plugin-sdk/v2@v2.33.0/helper/retry/state.go:86 +0x23b

Error: The terraform-provider-mongodbatlas_v1.15.3 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 1 month ago

Thanks for opening this issue! Please make sure you've followed our guidelines when opening the issue. In short, to help us reproduce the issue we need:

Terraform configuration file used to reproduce the issue
Terraform log files from the run where the issue occurred
Terraform Atlas provider version used to reproduce the issue
Terraform version used to reproduce the issue
Confirmation if Terraform OSS, Terraform Cloud, or Terraform Enterprise deployment

The ticket CLOUDP-248308 was created for internal tracking.

oarbusi commented 1 month ago

Thanks for opening the issue @gengdupuis . I have reproduced the issue. The issue happens because the database and collections you are specifying do not exist.

the cluster on which this occurs is empty, meaning that there are still no databases or collections existing in it. We would expect those to be created upon the index being created.

This assumption is not correct. Database and collections need to exists before creating the indexes. If not, it will fail.

I will close the issue because this is the expected behaviour. Feel free to open another one if you have any other issue. Thanks again

gengdupuis commented 1 month ago

Thanks for the quick reply @oarbusi. I assumed the collection would be created when the index is created, because this behavior is mentioned in the doc here. Same goes for databases. Shouldn't the resources created from terraform reflect the same behavior as what we expect when using mongosh?

gengdupuis commented 1 month ago

Also, do you have any guidelines as to how we should provision all our MongoDB resources as code? The best for my case would be to be able to create collections in terraform, but from what I have gathered, this is not possible. However, it has been discussed in the past here.

Zuhairahmed commented 1 month ago

Hi @gengdupuis, as you've noted, currently the Terraform Atlas Provider does not support creating/managing collections or databases. This is because this feature is not yet supported via the MongoDB Atlas Admin API which our Terraform Provider is built on top of (https://www.mongodb.com/docs/atlas/reference/api-resources-spec/v2/).

Definitely appreciate your interest in this functionality and understand how beneficial it would be to manage all resources as code. I encourage you to share your feedback or upvote this feature on our feedback platform at link you shared above. We value user input and is crucial in helping us prioritize new features and improvements going forward.

Lastly, in the example you shared above, because in Terraform we don't support the creation of collections/databases we aren't able to create those either via Terraform on your behalf when a user attempts to reference and they do not yet exist. Suggest for this workflow keep using via the the MongoDB Shell or Compass.

Please let me know if you need further assistance or have more questions. We're here to help!

mongodb / terraform-provider-mongodbatlas