Create alert with more than 1 notification

[Kikivsantos]

Hi, there.

I'm trying to create MongoDb Atlas alerts with terraform + terragrunt and I got a bug when trying to use more than 1 notification for the same alert.

Terraform CLI and Terraform MongoDB Atlas Provider Version

Terraform v1.5.3

Terraform Configuration File

main.tf

resource "mongodbatlas_alert_configuration" "default" {
  project_id = var.project_id #var.project_id[var.environment]
  event_type = var.event_type #"OUTSIDE_METRIC_THRESHOLD"
  enabled    = var.enabled

  matcher {
    field_name = try(var.matcher.field_name, null)
    operator   = try(var.matcher.operator, null)
    value = try(var.matcher.value, null)
  }

  metric_threshold_config {
    metric_name = try(var.metric_threshold_config.metric_name, null) #"ASSERT_REGULAR"
    operator    = try(var.metric_threshold_config.operator, null) #"LESS_THAN"
    threshold   = try(var.metric_threshold_config.threshold, null)#99.0
    units       = try(var.metric_threshold_config.units, null)#"RAW"
    mode        = try(var.metric_threshold_config.mode, null)#"AVERAGE"
  }

  threshold_config {
    operator    = try(var.threshold_config.operator, null) #"LESS_THAN"
    threshold   = try(var.threshold_config.threshold, null)#99.0
    units       = try(var.threshold_config.units, null)#"RAW"
  }

  notification {
    type_name     = var.notification.type_name #"GROUP" SLACK etc
    interval_min  = var.notification.interval_min # 5
    delay_min     = var.notification.delay_min #0
    sms_enabled   = try(var.notification.sms_enabled, null) #false
    email_enabled = try(var.notification.email_enabled, null) #true
    roles         = try(var.notification.roles, null) #["GROUP_OWNER", "GROUP_CLUSTER_MANAGER"]
    api_token     = try(var.slack_bot_token_dbre, null) 
    channel_name  = try(var.notification.channel_name, null)
    datadog_api_key = try(var.datadog_api_key,null)  
    datadog_region  = try(var.notification.datadog_region, null)
    email_address = try(var.notification.email_address, null)
    mobile_number   = try(var.notification.mobile_number, null)
    ops_genie_api_key = try(var.ops_genie_api_key, null)
    ops_genie_region  = try(var.notification.ops_genie_region, null)
    service_key       = try(var.service_key, null)
    username          = try(var.notification.username, null)
    victor_ops_api_key  = try(var.victor_ops_api_key, null)
    victor_ops_routing_key  = try(var.victor_ops_routing_key, null)
    webhook_secret          = try(var.webhook_secret, null)
    webhook_url             = try(var.notification.webhook_url, null)
    microsoft_teams_webhook_url = try(var.notification.microsoft_teams_webhook_url, null)
  }

}

variavle.tf

variable "project_id" {
    description = <<HEREDOC
    (Required) The ID of the project where the alert configuration will create.
    HEREDOC

}

variable "enabled" {
    description = <<HEREDOC
    It is not required, but If the attribute is omitted, by default will be false, and the configuration would be disabled. You must set true to enable the configuration.
    HEREDOC
    default = true
}

variable "event_type" {
    description = <<HEREDOC
    (Required) The type of event that will trigger an alert.
    OPtions in https://www.mongodb.com/docs/cloud-manager/reference/alert-types/#alert-event-types
    HEREDOC
}

variable "notification" {
  description = "List of notifications to send when an alert condition is detected"
  type = any
  default = {
    type_name               = "GROUP",
    channel_name            = null,
    datadog_api_key         = null,
    delay_min               = 0,
    interval_min            = 60,
    email_enabled           = null,
    sms_enabled             = null,
    mobile_number           = null,
    roles                   = null,
    datadog_region          = null,
    email_address           = null,
    ops_genie_region        = null,
    username                = null,
    webhook_url             = null,
    microsoft_teams_webhook_url = null
  }
}

variable "slack_bot_token_dbre" {
    type = string
    sensitive = true
}

variable "ops_genie_api_key" {
    type = string
    sensitive = true
    default = null
}

variable "victor_ops_api_key" {
  type = string
  sensitive = true
  default = null
}

variable "service_key" {
  type = string
  sensitive = true
  default = null
}

variable "victor_ops_routing_key" {
  type = string
  sensitive = true
  default = null
}

variable "datadog_api_key" {
    type = string
    sensitive = true  
    default = null
}

variable "webhook_secret" {
    type = string
    sensitive = true
    default = null
}

variable "metric_threshold_config" {
    description = <<HEREDOC
    The threshold that causes an alert to be triggered. 
    Required if event_type_name : OUTSIDE_METRIC_THRESHOLD or OUTSIDE_SERVERLESS_METRIC_THRESHOLD
    HEREDOC
    type = any
    default = null
}

variable "threshold_config" {
    type = any
    default = null
}

variable "matcher" {
    type = any
    default = null
}

terragrunt.hcl

locals {
  component_name = "modules/alert"
  component_version = "feature/changeAlert-slack"
}

include "root" {
  path = "${get_repo_root()}/terragrunt.hcl"
}

dependency "project" {
  config_path = "../.."
}

inputs = {
  project_id   = dependency.project.outputs.project_id
  event_type = "CLUSTER_MONGOS_IS_MISSING" 
  enabled   = false

  notification = {
    type_name     = "SLACK"
    interval_min  = 1440
    delay_min     = 0   
    #api_token = using as a environment variable
    channel_name = "database-alerts"
  }

  notification = {
    type_name     = "GROUP"
    interval_min  = 1440
    delay_min     = 0   
    sms_enabled   = false
    email_enabled = true
    roles         = ["GROUP_OWNER"] 
  }

} 

Steps to Reproduce

terragrunt plan
terragrunt apply

Expected Behavior

It should create one alert with 2 notification (1 for slack and 1 for group)

Actual Behavior

It creates only with the GROUP notification. Plan result below:

terragrunt plan

Initializing the backend...

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding latest version of mongodb/mongodbatlas...
- Installing mongodb/mongodbatlas v1.11.1...
- Installed mongodb/mongodbatlas v1.11.1 (signed by a HashiCorp partner, key ID 2A32ED1F3AD25ABF)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Acquiring state lock. This may take a few moments...

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # mongodbatlas_alert_configuration.default will be created
  + resource "mongodbatlas_alert_configuration" "default" {
      + alert_configuration_id = (known after apply)
      + created                = (known after apply)
      + enabled                = false
      + event_type             = "CLUSTER_MONGOS_IS_MISSING"
      + id                     = (known after apply)
      + project_id             = "5f6f9959a55ed91e80e4f7d1"
      + updated                = (known after apply)

      + matcher {}

      + metric_threshold_config {}

      + notification {
          + api_token     = (sensitive value)
          + delay_min     = 0
          + email_enabled = true
          + interval_min  = 1440
          + roles         = [
              + "GROUP_OWNER",
            ]
          + sms_enabled   = false
          + team_name     = (known after apply)
          + type_name     = "GROUP"
        }

      + threshold_config {}
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + EVENT_TYPE = "CLUSTER_MONGOS_IS_MISSING"

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Debug Output

Crash Output

Additional Context

References

[github-actions[bot]]

Thanks for opening this issue. The ticket INTMDB-1114 was created for internal tracking.

[maastha]

Hi @Kikivsantos This looks like an issue related how terragrunt is being used. I don't see any usage of constructs similar in functionality like for_each or dynamic blocks to add multiple blocks of notification in main.tf. I did come across https://terragrunt.gruntwork.io/docs/features/inputs/ as per which

If the same variable is assigned multiple values, Terraform will use the last value it finds overriding any previous values.

My guess is this is what is happening in this case where notification with type_name = "GROUP" is only included in the plan.

That said, we would request you to reproduce this issue only with Terraform and let us know if you get similar behavior in that case as well.

[Kikivsantos]

@maastha thank you for the answer. It did work using dynamic.