COLLECTION_ROLES_LIMIT_EXCEEDED - But not even close to the limit

Kikivsantos commented 1 month ago

Hi, I'm trying to execute a change in some of our users (to restric their access to only the collections that they need), but I'm getting the error below when doing it:

Plan: 0 to add, 1 to change, 0 to destroy.
mongodbatlas_database_user.default: Modifying... [id=YXV0aF9kYXRhYmFzZV9uYW1l:YWRtaW4=-cHJvamVjdF9pZA==:NWY2Zjk5NTlhNTVlZDkxZTgwZTRmN2Qx-dXNlcm5hbWU=:aW50ZXJvcF9kYXRhLXN5bmNfYmFzZWN0cmxfYXBp]
Error: error during database user creation
  with mongodbatlas_database_user.default,
  on main.tf line 35, in resource "mongodbatlas_database_user" "default":
  35: resource "mongodbatlas_database_user" "default" ***
https://cloud.mongodb.com/api/atlas/v2/groups/5f6f9959a55ed91e80e4f7d1/databaseUsers/admin/interop_data-sync_basectrl_api
PATCH: HTTP 403 Forbidden (Error code: "COLLECTION_ROLES_LIMIT_EXCEEDED")
Detail: Exceeded maximum number of collection level permissions. This group
can define at most 100 collection level roles. Reason: Forbidden. Params:
[100]
time=2024-05-23T16:29:39Z level=error msg=Terraform invocation failed in /home/gitrunner/actions-runner/_work/mongodb-atlas-automation/mongodb-atlas-automation/terraform/mongodb-atlas/clusters/receivables-230/users/interop_data-sync_basectrl_api/.terragrunt-cache/kvf9dosnAvuaqcrksx5t2MbKmvM/pzZ6kKKOog-nM-3feLfydekfdws/modules/application-users prefix=[terraform/mongodb-atlas/clusters/receivables-230/users/interop_data-sync_basectrl_api] 
time=2024-05-23T16:29:39Z level=error msg=1 error occurred:
    * exit status 1

It says that the limit is 100 but i'm not even close to this number and getting the error. I'm addind the access to 9 collections.

Terraform CLI and Terraform MongoDB Atlas Provider Version

latest

Initializing provider plugins...
- Finding latest version of hashicorp/random...
- Finding latest version of hashicorp/google...
- Finding latest version of mongodb/mongodbatlas...
- Installing hashicorp/random v3.6.2...
- Installed hashicorp/random v3.6.2 (signed by HashiCorp)
- Installing hashicorp/google v5.30.0...
- Installed hashicorp/google v5.30.0 (signed by HashiCorp)
- Installing mongodb/mongodbatlas v1.16.0...
- Installed mongodb/mongodbatlas v1.16.0

# Copy-paste your version.tf and provider.tf (or equivalent) here

#  # ------------------------------------------------------------------------------
#  # TERRAGRUNT CONFIGURATION
#  # ------------------------------------------------------------------------------

terraform {
  source               = "${local.blueprint_repository}//${local.component_name}?ref=${local.component_version}"
}

locals {
  repo_root            = run_cmd("--terragrunt-quiet", "git", "rev-parse", "--show-toplevel")
  blueprint_repository = "git::https://github.com/tag-trade-repository/tf-module-atlas.git"
  component_name       = tostring(run_cmd("--terragrunt-quiet", "${local.repo_root}/_bin/read-component-value.sh", "component_name", "${local.repo_root}/${path_relative_to_include()}/"))
  component_version    = try(tostring(run_cmd("--terragrunt-quiet", "${local.repo_root}/_bin/read-component-value.sh", "component_version", "${local.repo_root}/${path_relative_to_include()}/")), "latest")
}

# Generate the remote state block
remote_state {
  backend = "gcs"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }
  config = {
    bucket = "gcs-db-services-tfstate"
    prefix = "${path_relative_to_include()}/terraform.tfstate"
  }
}

# Generate the provider block
generate "provider" {
  path = "provider.tf"
  if_exists = "overwrite"
  contents = <<EOF
    provider "mongodbatlas" {}
    provider "google" {
        project = "eng-receivables-dev-r202103"
        region  = "us-central1"
    }
EOF
} 

# Generate the version block
generate "versions" {
  path = "versions.tf"
  if_exists = "overwrite"
  contents = <<EOF
    terraform {
      required_providers {
        mongodbatlas = {
          source = "mongodb/mongodbatlas"
        }
        google = {
          source = "hashicorp/google"
        }
      }
    }
EOF
}

Terraform Configuration File

Terraform (module):

MAIN:

locals {
    project_name              = "${terraform.workspace}" == "master" ? "production" : "${terraform.workspace}"

    project                   = {
        engineering           = {
            "develop"         = "158169389750"
            "staging"         = "120608605102"
            "master"          = "138405083369"
        }
        database              = {
            "develop"         = "321244247844"
            "staging"         = "903367255309"
            "master"          = "861528625251"
        }
    }

    env                       = {
        master                = "prd",
        staging               = "stg",
        develop               = "dev"
    }
}

# ------------------------------------------------------------------------------
# RANDOM PASSWORD
# ------------------------------------------------------------------------------
resource "random_password" "default" {
    length                    = var.password_length
    special                   = false
}

# ------------------------------------------------------------------------------
# DATABASE USER
# ------------------------------------------------------------------------------
resource "mongodbatlas_database_user" "default" {
    project_id                = data.mongodbatlas_project.default.id
    username                  = var.username
    password                  = random_password.default.result
    auth_database_name        = var.auth_database_name

    dynamic "roles" {
        for_each              = var.roles
        content {
            role_name         = roles.value["role_name"]
            database_name     = roles.value["database_name"]
            collection_name   = try(roles.value["collection_name"], null)
        }
    }

    scopes {
        name                  = var.old_cluster_bool ? var.scope_name[local.env[terraform.workspace]].value : "mgo-${local.env[terraform.workspace]}-${var.cluster_name}" 
        #name                  = var.scope_name[local.env[terraform.workspace]].value
        type                  = var.scope_type
    }
}

# ------------------------------------------------------------------------------
# GCP SECRET MANAGER
# ------------------------------------------------------------------------------
resource "google_secret_manager_secret" "default" {
    project                   = "${lookup(local.project[var.secret_project_name], "${terraform.workspace}")}"
    secret_id                 = local.secret_id
    labels                    = var.secret_labels
    replication {
        auto {}
    }
}

# ------------------------------------------------------------------------------
# GCP SECRET VERSION
# ------------------------------------------------------------------------------
resource "google_secret_manager_secret_version" "default" {
    secret                    = google_secret_manager_secret.default.id
    secret_data               = local.private_connection_string
}

# ------------------------------------------------------------------------------
# CREATE PRIVATE CONNECTION STRING FOR SECRET MANAGER
# ------------------------------------------------------------------------------
locals {
    database_name             = lookup(var.roles[0], "database_name")
    auth_database_name        = local.database_name == null ? var.auth_database_name : local.database_name
    cluster_uri               = data.mongodbatlas_advanced_cluster.default.connection_strings == null ? "connection_string" : trimprefix(data.mongodbatlas_advanced_cluster.default.connection_strings.0.private_srv, "mongodb+srv://")
    password                  = random_password.default.result
    private_connection_string = "mongodb+srv://${mongodbatlas_database_user.default.username}:${local.password}@${local.cluster_uri}/${local.auth_database_name}?retryWrites=true&w=majority${var.readPreference}"
    username_split            = split("-",mongodbatlas_database_user.default.username)
    username_replace          = replace(local.username_split[1], "_", "")
    secret_id                 = replace("MONGO_DB_${upper(local.auth_database_name)}_USER_${upper(local.username_replace)}_CONNECTION_STRING", "-", "")
}

VARIABLE:

# ------------------------------------------------------------------------------
# VARIABLES
# ------------------------------------------------------------------------------
# ------------------------------------------------------------------------------
# MONGODB DATABASE USERS
# ------------------------------------------------------------------------------
variable "username" {
    description = <<HEREDOC
    Required - The name of database user.
    HEREDOC
}
variable "password" {
    description = <<HEREDOC
    Optional - The password of database user.
    HEREDOC
    default     = null
}
variable "password_length" {
    description = <<HEREDOC
    Optional - The length of password of database user.
    HEREDOC
    default     = 20
}
variable "auth_database_name" {
    description = <<HEREDOC
    Optional - Database against which mongodb atlas authenticates the user. 
    HEREDOC
    default     = "admin"
}
variable "roles" {
    description = <<HEREDOC
    Required - One or more user roles blocks.
    HEREDOC
    type        = list(map(string))
}

variable "old_cluster_bool" {
    description = <<HEREDOC
    Optional - Specifies if the cluster is an old cluster.
    HEREDOC
    type        = bool
    default     = true
}

variable "cluster_name" {
    description = <<HEREDOC
    optional - specifies the cluster name
    HEREDOC
    type = string
    default = null
}

variable "scope_name" {
    description = <<HEREDOC
    Required - One or more user cluster scopes blocks.
    HEREDOC
    type        = map(object({
        value = string
    }))
    default = {}
}
variable "scope_type" {
    description = <<HEREDOC
    Required - One or more user cluster scopes blocks.
    HEREDOC
    type        = string
}
variable "secret_project_name" {
    description = <<HEREDOC
    Optional - The name of the GCP project in which the resource belongs.
    If it is not provided, the provider project is used.
    HEREDOC
    default = "engineering"
}
variable "secret_labels" {
    description = <<HEREDOC
    Optional - The labels assigned to this Secret. 
    - Label keys must be between 1 and 63 characters long, have a UTF-8 encoding of
      maximum 128 bytes, and must conform to the following PCRE regular expression: 
        [\p{Ll}\p{Lo}][\p{Ll}\p{Lo}\p{N}-]{0,62} 
    - Label values must be between 0 and 63 characters long, have a UTF-8 encoding
      of maximum 128 bytes, and must conform to the following PCRE regular expression:
         [\p{Ll}\p{Lo}\p{N}-]{0,63}
    - No more than 64 labels can be assigned to a given resource. An object containing
      a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg"}.
    HEREDOC
    default     = {}
}

variable "readPreference" {
  description = <<HEREDOC
  Informa caso queira que a leitura seja feita no nó secundário
  HEREDOC
  type = string
  default = ""
}

Steps to Reproduce

terragrunt plan
=> Runs ok

terraform apply
=> Returns error

Expected Behavior

It should change the user adding the access to the 9 collections

Actual Behavior

Plan: 0 to add, 1 to change, 0 to destroy.
mongodbatlas_database_user.default: Modifying... [id=YXV0aF9kYXRhYmFzZV9uYW1l:YWRtaW4=-cHJvamVjdF9pZA==:NWY2Zjk5NTlhNTVlZDkxZTgwZTRmN2Qx-dXNlcm5hbWU=:aW50ZXJvcF9kYXRhLXN5bmNfYmFzZWN0cmxfYXBp]
Error: error during database user creation
  with mongodbatlas_database_user.default,
  on main.tf line 35, in resource "mongodbatlas_database_user" "default":
  35: resource "mongodbatlas_database_user" "default" ***
https://cloud.mongodb.com/api/atlas/v2/groups/5f6f9959a55ed91e80e4f7d1/databaseUsers/admin/interop_data-sync_basectrl_api
PATCH: HTTP 403 Forbidden (Error code: "COLLECTION_ROLES_LIMIT_EXCEEDED")
Detail: Exceeded maximum number of collection level permissions. This group
can define at most 100 collection level roles. Reason: Forbidden. Params:
[100]
time=2024-05-23T16:29:39Z level=error msg=Terraform invocation failed in /home/gitrunner/actions-runner/_work/mongodb-atlas-automation/mongodb-atlas-automation/terraform/mongodb-atlas/clusters/receivables-230/users/interop_data-sync_basectrl_api/.terragrunt-cache/kvf9dosnAvuaqcrksx5t2MbKmvM/pzZ6kKKOog-nM-3feLfydekfdws/modules/application-users prefix=[terraform/mongodb-atlas/clusters/receivables-230/users/interop_data-sync_basectrl_api] 
time=2024-05-23T16:29:39Z level=error msg=1 error occurred:
    * exit status 1

github-actions[bot] commented 1 month ago

Thanks for opening this issue! Please make sure you've followed our guidelines when opening the issue. In short, to help us reproduce the issue we need:

Terraform configuration file used to reproduce the issue
Terraform log files from the run where the issue occurred
Terraform Atlas provider version used to reproduce the issue
Terraform version used to reproduce the issue
Confirmation if Terraform OSS, Terraform Cloud, or Terraform Enterprise deployment

The ticket CLOUDP-250416 was created for internal tracking.

Kikivsantos commented 1 month ago

I'm sorry, I didn't add the code that I was running (that calls the module):

Terragrunt:

include {
  path = find_in_parent_folders()
}
locals {
  component_name = "modules/application-users"
  component_version = "v1.3.6"
  cluster_vars = read_terragrunt_config(find_in_parent_folders("cluster.hcl")).locals
}
inputs = {
  username = "interop_data-sync_basectrl_api"
  roles = [

    {
      database_name = "interop-data"
      role_name = "readWrite"

      collection_name = "acquirer"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "acquirer-payment-scheme"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "commercial-establishment-acquirer"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "commercial-establishment"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "consent"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "creditor"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "optout-tag-basectrl"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "slc-participant"
    },
    {
      database_name = "interop-data"
      role_name = "readWrite"
      collection_name = "payment-scheme"
    },

  ]

  scope_name = local.cluster_vars.old_cluster
  scope_type = "CLUSTER"
  labels = "interop_data-sync_basectrl_api"
}

Kikivsantos commented 1 month ago

Seems to be a limit on Atlas and not a problem with terraform.

Sorry about that

mongodb / terraform-provider-mongodbatlas