terraform-google-modules / terraform-google-project-factory

Creates an opinionated Google Cloud project by using Shared VPC, IAM, and Google Cloud APIs
https://registry.terraform.io/modules/terraform-google-modules/project-factory/google
Apache License 2.0
825 stars 533 forks source link

Unable to add multiple Quotas for the same metric. #890

Closed amerenda closed 3 months ago

amerenda commented 5 months ago

TL;DR

Trying to add the following consumer_quotas:

consumer_quotas = [
    {
      service    = "bigquery.googleapis.com"
      metric     = urlencode("bigquery.googleapis.com/quota/query/usage")
      limit      = urlencode("/d/project")
      value      = "2097152"
      dimensions = {}
    },
    {
      service    = "bigquery.googleapis.com"
      metric     = urlencode("bigquery.googleapis.com/quota/query/usage")
      limit      = urlencode("/d/project/user")
      value      = "2097152"
      dimensions = {}
    },
  ]

I get this error:

  │ Error: Duplicate object key
│
│   on .terraform/modules/data-pipelines/modules/quota_manager/main.tf line 18, in locals:
│   18:   consumer_quotas = { for index, quota in var.consumer_quotas : "${quota.service}-${quota.metric}" => quota }
│     ├────────────────
│     │ quota.metric is "bigquery.googleapis.com%2Fquota%2Fquery%2Fusage"
│     │ quota.service is "bigquery.googleapis.com"
│
│ Two different items produced the key "bigquery.googleapis.com-bigquery.googleapis.com%2Fquota%2Fquery%2Fusage" in this 'for' expression. If duplicates are expected, use the ellipsis (...) after the value expression to enable grouping by key.

The map consumer_quotas is built using the service and metric to create a unique name, but since the Quotas Query usage per day per user and Query usage per share the same metric (bigquery.googleapis.com/quota/query/usage) I am unable to add both.

locals {
  consumer_quotas = { for index, quota in var.consumer_quotas : "${quota.service}-${quota.metric}" => quota }
}

Expected behavior

Two quota resources get created

Observed behavior

User
│ Error: Duplicate object key
│
│   on .terraform/modules/data-pipelines/modules/quota_manager/main.tf line 18, in locals:
│   18:   consumer_quotas = { for index, quota in var.consumer_quotas : "${quota.service}-${quota.metric}" => quota }
│     ├────────────────
│     │ quota.metric is "bigquery.googleapis.com%2Fquota%2Fquery%2Fusage"
│     │ quota.service is "bigquery.googleapis.com"
│
│ Two different items produced the key "bigquery.googleapis.com-bigquery.googleapis.com%2Fquota%2Fquery%2Fusage" in this 'for' expression. If duplicates are expected, use the ellipsis (...) after the value expression to enable grouping by key.

Terraform Configuration

module "data-pipelines" {
  source  = "terraform-google-modules/project-factory/google"
  version = "~> 14.5"

  name                    = "PROJECT_ID"
  random_project_id       = false
  org_id                  = var.org_id
  billing_account         = var.billing_account
  folder_id               = var.folder_id
  svpc_host_project_id    = module.network.project_id
  shared_vpc_subnets      = []
  auto_create_network     = false
  default_service_account = "disable"
  create_project_sa       = false
  activate_apis           = []

  labels = {
    environment = var.environment
  }

  consumer_quotas = [
    {
      service    = "bigquery.googleapis.com"
      metric     = urlencode("bigquery.googleapis.com/quota/query/usage")
      limit      = urlencode("/d/project")
      value      = "2097152"
      dimensions = {}
    },
    {
      service    = "bigquery.googleapis.com"
      metric     = urlencode("bigquery.googleapis.com/quota/query/usage")
      limit      = urlencode("/d/project/user")
      value      = "2097152"
      dimensions = {}
    },
  ]
}

### Terraform Version

```sh
Terraform v1.5.7

Additional information

No response

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days