terraform-google-modules / terraform-google-sql-db

Creates a Cloud SQL database instance
https://registry.terraform.io/modules/terraform-google-modules/sql-db/google
Apache License 2.0
263 stars 422 forks source link

feat: Add support to use existing notification channels for alerts #593

Closed ps-occrp closed 4 months ago

ps-occrp commented 4 months ago

Add support to use existing notification channels for alerts

imrannayer commented 4 months ago

@ps-occrp can u also add a notification channel and pass it to the module in this example so it will become part of the test.

imrannayer commented 4 months ago

/gcbrun

imrannayer commented 4 months ago

@ps-occrp integration test is failing with following error:

estPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: Error: Failed to retrieve project, pid: , err: project: required field is not set
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   with google_monitoring_notification_channel.email,
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   48: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: Error: Failed to retrieve project, pid: , err: project: required field is not set
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   58: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Failed to retrieve project, pid: , err: project: required field is not set

  with google_monitoring_notification_channel.email,
  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
  48: resource "google_monitoring_notification_channel" "email" {

Error: Failed to retrieve project, pid: , err: project: required field is not set

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
            Error Trace:    /builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
            Error:          Received unexpected error:
                            FatalError{Underlying: error while running command: exit status 1; 
                            Error: Failed to retrieve project, pid: , err: project: required field is not set

                              with google_monitoring_notification_channel.email,
                              on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
                              48: resource "google_monitoring_notification_channel" "email" {

                            Error: Failed to retrieve project, pid: , err: project: required field is not set

                              with module.backup.google_monitoring_notification_channel.email[0],
                              on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
                              58: resource "google_monitoring_notification_channel" "email" {
                            }
            Test:           TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/18 14:31:39 RUN_STAGE env var set to apply
ps-occrp commented 4 months ago

@imrannayer Can you run gcbrun again?

imrannayer commented 4 months ago

/gcbrun

imrannayer commented 4 months ago

@ps-occrp test failed:

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-19T14:36:39Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Failed to retrieve project, pid: , err: project: required field is not set

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
            Error Trace:    /builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
            Error:          Received unexpected error:
                            FatalError{Underlying: error while running command: exit status 1; 
                            Error: Failed to retrieve project, pid: , err: project: required field is not set

                              with module.backup.google_monitoring_notification_channel.email[0],
                              on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
                              58: resource "google_monitoring_notification_channel" "email" {
                            }
            Test:           TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/19 14:36:39 RUN_STAGE env var set to apply
2024/04/19 14:36:39 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (61.60s)
FAIL
FAIL    github.com/terraform-google-modules/terraform-google-sql-db/test/integration/postgresql-backup-provided-service-account 62.358s
ps-occrp commented 4 months ago

@imrannayer can you run gcbrun again?

imrannayer commented 4 months ago

@ps-occrp failing linit test

imrannayer commented 4 months ago

/gcbrun

ps-occrp commented 4 months ago

@imrannayer looks like gcbrun failed, can you paste error message?

imrannayer commented 4 months ago

@ps-occrp

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:50Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Still creating... [10s elapsed]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Creation complete after 14s [id=projects/ci-sql-db-c35d/locations/us-central1/jobs/sql-backup-example-postgres-e6e8c636]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Details:
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.Help",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "links": [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "description": "Google developers console API activation",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "domain": "googleapis.com",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "metadata": {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "consumer": "projects/972442344775",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "service": "monitoring.googleapis.com"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "reason": "SERVICE_DISABLED"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   with google_monitoring_notification_channel.email,
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   48: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Details:
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.Help",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "links": [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "description": "Google developers console API activation",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "domain": "googleapis.com",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "metadata": {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "consumer": "projects/972442344775",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "service": "monitoring.googleapis.com"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "reason": "SERVICE_DISABLED"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   58: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Google developers console API activation",
        "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
      }
    ]
  },
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/972442344775",
      "service": "monitoring.googleapis.com"
    },
    "reason": "SERVICE_DISABLED"
  }
]

  with google_monitoring_notification_channel.email,
  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
  48: resource "google_monitoring_notification_channel" "email" {

Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Google developers console API activation",
        "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
      }
    ]
  },
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/972442344775",
      "service": "monitoring.googleapis.com"
    },
    "reason": "SERVICE_DISABLED"
  }
]

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
            Error Trace:    /builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
            Error:          Received unexpected error:
                            FatalError{Underlying: error while running command: exit status 1; 
                            Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
                            Details:
                            [
                              {
                                "@type": "type.googleapis.com/google.rpc.Help",
                                "links": [
                                  {
                                    "description": "Google developers console API activation",
                                    "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
                                  }
                                ]
                              },
                              {
                                "@type": "type.googleapis.com/google.rpc.ErrorInfo",
                                "domain": "googleapis.com",
                                "metadata": {
                                  "consumer": "projects/972442344775",
                                  "service": "monitoring.googleapis.com"
                                },
                                "reason": "SERVICE_DISABLED"
                              }
                            ]

                              with google_monitoring_notification_channel.email,
                              on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
                              48: resource "google_monitoring_notification_channel" "email" {

                            Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
                            Details:
                            [
                              {
                                "@type": "type.googleapis.com/google.rpc.Help",
                                "links": [
                                  {
                                    "description": "Google developers console API activation",
                                    "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
                                  }
                                ]
                              },
                              {
                                "@type": "type.googleapis.com/google.rpc.ErrorInfo",
                                "domain": "googleapis.com",
                                "metadata": {
                                  "consumer": "projects/972442344775",
                                  "service": "monitoring.googleapis.com"
                                },
                                "reason": "SERVICE_DISABLED"
                              }
                            ]

                              with module.backup.google_monitoring_notification_channel.email[0],
                              on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
                              58: resource "google_monitoring_notification_channel" "email" {
                            }
            Test:           TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/23 17:36:55 RUN_STAGE env var set to apply
2024/04/23 17:36:55 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (302.21s)
ps-occrp commented 4 months ago

@imrannayer Example creates a notification channel but looks like monitoring.googleapis.com is not enabled for integration test project. I don't think enabling api should be added to example.

imrannayer commented 4 months ago

@imrannayer Example creates a notification channel but looks like monitoring.googleapis.com is not enabled for integration test project. I don't think enabling api should be added to example.

Thats part of project setup. Can u plz add the API?

imrannayer commented 4 months ago

/gcbrun

imrannayer commented 4 months ago

/gcbrun

imrannayer commented 4 months ago

failed with following errors:

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:34Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Still creating... [10s elapsed]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Creation complete after 11s [id=projects/ci-sql-db-6063/locations/us-central1/jobs/sql-backup-example-postgres-0f5349ef]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   with google_monitoring_notification_channel.email,
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   48: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   58: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).

  with google_monitoring_notification_channel.email,
  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
  48: resource "google_monitoring_notification_channel" "email" {

Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
            Error Trace:    /builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
            Error:          Received unexpected error:
                            FatalError{Underlying: error while running command: exit status 1; 
                            Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).

                              with google_monitoring_notification_channel.email,
                              on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
                              48: resource "google_monitoring_notification_channel" "email" {

                            Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).

                              with module.backup.google_monitoring_notification_channel.email[0],
                              on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
                              58: resource "google_monitoring_notification_channel" "email" {
                            }
            Test:           TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/26 18:02:36 RUN_STAGE env var set to apply
2024/04/26 18:02:36 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (313.75s)
FAIL
ps-occrp commented 4 months ago

@imrannayer this looks like permission issues while creating notification channel. I'm not sure how I can reproduce this or where to look for the issue/fix.

imrannayer commented 4 months ago

/gcbrun

imrannayer commented 4 months ago

@ps-occrp can u plz add following permission in IAM file

roles/monitoring.editor

ps-occrp commented 4 months ago

@imrannayer Thanks, I am not familiar with test suit so I didn't know about this. For future work this will come in handy :)

imrannayer commented 4 months ago

/gcbrun

imrannayer commented 4 months ago

@ps-occrp

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Creation complete after 10s [id=projects/ci-sql-db-25d4/locations/us-central1/jobs/sql-backup-example-postgres-49b11252]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 400: Request was missing field notification_channel.labels[email_address].
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185:   on ../../modules/backup/main.tf line 66, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185:   66: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Error creating NotificationChannel: googleapi: Error 400: Request was missing field notification_channel.labels[email_address].

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 66, in resource "google_monitoring_notification_channel" "email":
  66: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
            Error Trace:    /builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:518
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:577
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:616
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/utils/stages.go:31
                                        /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:616
            Error:          Received unexpected error:
                            FatalError{Underlying: error while running command: exit status 1; 
                            Error: Error creating NotificationChannel: googleapi: Error 400: Request was missing field notification_channel.labels[email_address].

                              with module.backup.google_monitoring_notification_channel.email[0],
                              on ../../modules/backup/main.tf line 66, in resource "google_monitoring_notification_channel" "email":
                              66: resource "google_monitoring_notification_channel" "email" {
                            }
            Test:           TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/30 16:18:26 RUN_STAGE env var set to apply
2024/04/30 16:18:26 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (295.13s)
FAIL
FAIL    github.com/terraform-google-modules/terraform-google-sql-db/test/integration/postgresql-backup-provided-service-account 295.999s
FAIL
imrannayer commented 4 months ago

@ps-occrp you need to setup create_notification_channel to false. Thats causing integration test to fail.

imrannayer commented 4 months ago

/gcbrun

imrannayer commented 4 months ago

/gcbrun