Mongo collection not supporting unique field on _id index

[landintrees]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.4

AzureRM Provider Version

3.91.0

Affected Resource(s)/Data Source(s)

azurerm_cosmosdb_mongo_collection

Terraform Configuration Files

# cosmos account is mongodb version 3.6
data "azurerm_cosmosdb_account" "engineering" {
  name                = "test-cosmos-account"
  resource_group_name = "cosmos-test-rg"
}

resource "azurerm_cosmosdb_mongo_database" "mytestdb" {
  name                = "my-test-db"
  resource_group_name = "cosmos-test-rg"
  account_name        = data.azurerm_cosmosdb_account.engineering.name

  autoscale_settings {
    max_throughput = 4000
  }
}

resource "azurerm_cosmosdb_mongo_collection" "testfail" {
  name                = "testfail"
  resource_group_name = "cosmos-test-rg"
  account_name        = data.azurerm_cosmosdb_account.engineering.name
  database_name       = azurerm_cosmosdb_mongo_database.mytestdb.name

  depends_on          = [ azurerm_cosmosdb_mongo_database.mytestdb ]

  default_ttl_seconds = "-1"

 index {
      keys = [
        "$**"
      ]
  }

  index {
      keys = [
        "_id"

      ]
      unique = true
  }
  index {
      keys = [
        var.cosmosdb_shard_key
      ]
      unique = true
  }
  shard_key = var.cosmosdb_shard_key
}

Debug Output/Panic Output

module.arcnode.azurerm_cosmosdb_mongo_collection.testfail: Creating...
module.arcnode.azurerm_cosmosdb_mongo_collection.testfail: Still creating... [10s elapsed]
module.arcnode.azurerm_cosmosdb_mongo_collection.testfail: Still creating... [20s elapsed]
module.arcnode.azurerm_cosmosdb_mongo_collection.testfail: Still creating... [30s elapsed]
╷
│ Error: waiting on create/update future for Mongodb Collection: (Collection Name "testfail" / Mongodb Database Name "my-test-db" / Database Account Name "test-cosmos-account" / Resource Group "cosmos-test-rg"): Code="BadRequest" Message="The field 'unique' is not valid for _id index specification.\r\nActivityId: xxxxxxx-xxxxxx-xxxxx-xxxxxxx, Microsoft.Azure.Documents.Common/2.14.0, Microsoft.Azure.Documents.Common/2.14.0, Microsoft.Azure.Documents.Common/2.14.0, Microsoft.Azure.Documents.Common/2.14.0"

Expected Behaviour

Collection should be created successfully OR the docs should be updated so that unique=true is not required for indexes with _id key.

Actual Behaviour

Fails with error: Code="BadRequest" Message="The field 'unique' is not valid for _id index specification."

Steps to Reproduce

terraform apply

Important Factoids

We were able to create the collection by removing unique = true from the _id index. When added back in and re-running terraform apply there were no changes.

This is on version 3.6 of cosmos DB.

References

No response

[landintrees]

Note that this only started failing sometime after 5th Feb 2024 as we have been running this same plan regularly before that date.

[landintrees]

Our workaround is to remove unique = true from the _id index. This allows the terraform apply to complete successfully but future applies continuously nag with the following change:

      - index {
          - keys   = [
              - "_id",
            ] -> null
          - unique = true -> null
        }
      + index {
          + keys   = [
              + "_id",
            ]
          + unique = false
        }

However this does not actually change anything when applied.

According to the Microsoft docs uniqueness is enforced on _id anyway.

[rcskosir]

@landintrees Thanks for taking the time to open this issue. I believe this PR in addition to the service team's fix mentioned in the description will resolve this issue, #24817

[M3LiNdRu]

Hi @landintrees, I am really grateful that you spotted it. How did you proceed with the workaround? Did you update the tfstate manually as well?

[landintrees]

Hi @M3LiNdRu, we simple removed the "unique = true" from the _id index:

  index {
      keys = [
        "_id"
      ]
      unique = true     <-- removed this line
  }

As mentioned above terraform adds "unique = false" back on every apply but this makes no difference to the actual index in cosmos (we've confirmed this). It does make the plans a bit messy if you have a lot of collections but we can put up with it for now until it's fixed upstream.

We didn't update the tfstate, just ran terraform apply.

[rcskosir]

@landintrees Thanks for taking the time to open this issue. I believe this issue should be resolved by a roll back in the breaking change on Azure's side as noted in https://github.com/hashicorp/terraform-provider-azurerm/pull/24817#issuecomment-1984970946. As such I am going to close this issue, if you find that is not the case, please reach out and we can reopen.

[arm749]

@rcskosir I updated my ARM version to latest and I am still seeing this issue. I am not sure that whatever they rolled back corrected it.

[frmunozp]

@rcskosir I updated my ARM version to latest and I am still seeing this issue. I am not sure that whatever they rolled back corrected it.

I see the same error using Bicep version 0.26.54

[panfilenok-epam]

The issue is still there

[fleyer]

same here

[waarg]

We are still experiencing this on creates. Seems fine on updates.

[hilariocoelho]

Still experiencing this issue. Keep seeing changes on _id index even though there are no changes being made on apply.

[rcskosir]

I can reopen this issue since the behavior seems to still be occurring. @wuxu92 Are you able to provide any insight here on why this does not appear to be fixed by the roll back mentioned in https://github.com/hashicorp/terraform-provider-azurerm/pull/24817#issuecomment-1984970946?

[BosBer]

We also keep seeing changes while no changes are being made. Besides the clutter, this is for us causing apply executions while they are not required.

[waarg]

We are seeing the same as @BosBer