[bug] datadog_gcp_integration not working properly / incorrect documentation

sgringwe commented 4 years ago

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

0.12.18

Affected Resource(s)

datadog_integration_gcp

Terraform Configuration Files

resource "google_service_account" "datadog_insights_monitoring_account" {
  account_id   = "datadog-insights-monitoring"
  display_name = "Datadog Monitoring"
  description  = "A Service Account that is configured into Datadog to support GCP monitoring integrations"
}

resource "google_project_iam_member" "datadog_monitoring_account_metrics_viewer" {
  project = var.project
  role    = "roles/monitoring.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

resource "google_project_iam_member" "datadog_daemon_set_compute_instance_getter" {
  project = var.project
  role    = "roles/compute.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

resource "google_project_iam_member" "datadog_monitoring_account_cloud_asset_viewer" {
  project = var.project
  role    = "roles/cloudasset.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

## GCP/Datadog integration
resource "google_service_account_key" "datadog_insights_monitoring" {
  service_account_id = google_service_account.datadog_insights_monitoring_account.name
  public_key_type    = "TYPE_X509_PEM_FILE"
}

resource "datadog_integration_gcp" "gcp_project_integration" {
  project_id = var.project

  private_key_id = jsondecode(base64decode(google_service_account_key.datadog_insights_monitoring.private_key))["private_key_id"]
  private_key    = jsondecode(base64decode(google_service_account_key.datadog_insights_monitoring.private_key))["private_key"]

  client_email = google_service_account.datadog_insights_monitoring_account.email
  client_id    = google_service_account.datadog_insights_monitoring_account.unique_id
}

Debug Output

Please provider a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

The apply "succeeds".

Panic Output

If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.

Expected Behavior

What should have happened?

Integration is working

Actual Behavior

What actually happened?

Integration is not working. Integrations UI shows error message "- Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions"

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

terraform apply

Important Factoids

n/a

References

n/a

jirikuncar commented 4 years ago

@sgringwe can you please open a support ticket or provide more details? Thank you!

sgringwe commented 4 years ago

@jirikuncar what additional details would you like to see?

jirikuncar commented 4 years ago

It's a bit difficult for me to debug the problem without the request that was made. We can help you through the support ticket to investigate the real cause of this problem and if it happens to be terraform-provider-datadog we will have more details then. Thank you for your understanding.

jskowalski commented 4 years ago

I'm having the same issue. Complete the google service account, upload file (success) then errors with "- Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions"

kassim commented 3 years ago

I'm experiencing the same issue/error warning when trying to set up the datadog gcp integration. Is there a way to debug or solve?

davidtom commented 3 years ago

Encountering this issue as well, which is blocking my team from adopting Datadog. Would love to have a solution for this.

kassim commented 3 years ago

Actually I came back to it the day after and it seemed to be working.

Could be GCP took a while to make permissions work.. Or maybe someone in my huge organisation fixed something with our config and I wasn't aware.

On Thu, 24 Dec 2020, 17:42 David Tomczyk, notifications@github.com wrote:

Encountering this issue as well, which is blocking my team from adopting Datadog. Would love to have a solution for this.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DataDog/terraform-provider-datadog/issues/464#issuecomment-750924647, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUCP2DK4S66P6AINSHYHULSWNVNVANCNFSM4LW2ZN2A .

davidtom commented 3 years ago

Actually I came back to it the day after and it seemed to be working. Could be GCP took a while to make permissions work.. Or maybe someone in my huge organisation fixed something with our config and I wasn't aware.

Interesting. Was wondering if that was the issue, but waiting for the permissions didn't seem to change anything for us.

However, I was able to get it working using the following roles for the service account:

resource "google_project_iam_member" "service_account" {
  for_each = toset([
    "roles/compute.viewer",
    "roles/container.viewer",
    "roles/monitoring.viewer",
  ])

  project = var.project
  role    = each.value
  member  = "serviceAccount:${google_service_account.datadog.email}"
}

I got those from this module.

Hopefully this can help someone, but very curious why this works even though its quite different than what is in the documentation.

nmuesch commented 3 years ago

Hey, apologies for the delay here. I wanted to check in and see if you were still experiencing an issue with this. Would you also mind pointing to the documentation you're referring to?

Looking at the documentation here - https://docs.datadoghq.com/integrations/google_cloud_platform/?tab=datadogussite#installation the list of roles seem to be present.

markus-wa commented 3 years ago

We're seeing a similar issue. On Friday our GCP integration stopped working, seemingly out of the blue. Monday we rotated the associated GCP SA Key and updated the Terraform resource gcp_project_integration which was successful - but this didn't help.

the service account has the appropriate permissions

jtafurth commented 3 years ago

I'm also having the same problem

Stopped working last week, I updated the provider to the latest version but still doesn't work, I do get data back from google but it seems to be working intermittently.

xavidop commented 2 years ago

Hi we are facing the same issue! I just followed this configuration: https://github.com/nephosolutions/terraform-google-datadog-integration/blob/master/main.tf#L17

and using the terraform resource its not working, in fact, when I go to that service account and I export the json file manually and then I import it on datatadog manually as well, it works.

Really really weird, I am using the datadog provider 3.4.0

sergio-guillen-exa commented 2 years ago

I am also having the same error but not only with the Datadog Terraform provider. What I tested so far without success:

GCP project with billing, compute, monitoring, cloud asset services enabled
Service account that has viewer roles for compute, monitoring and cloud asset (one service account created with Terraform and the other created manually)
Credentials created with terraform and manually
Datadog integration keeps having the same error using Terraform resource, manually creating using Terraform service account and manually creating using the manually-created service account.

The versions I used for the provider are 3.4.0 and 3.2.0 (this was using for other projects)

Is it possible that the GCP project requires another service to be enabled?

ajmunoz411 commented 1 year ago

I'm also having this same issue and haven't yet been able to find a fix. I've been using the GCP integration for over 10 months without issue but today started having this issue. It was right after I tried adding a host_filter that the integration started showing up as Broken with the Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions error. So far, I've tried:

creating the service account (with the viewer roles for compute, monitoring, and cloud asset) both manually and via terraform
tried the alternate roles suggested by @davidtom above
I'm on on version 3.12.0 for the DataDog provider

Still no luck so far, wondering if there are any other things to try to resolve this?

c-p-b commented 1 year ago

I was able to get this working manually, basically by removing any automations created by terraform and then manually creating a key in GCP UI, then manually adding that json file into Datadog UI. I had terraform create the service account with these permissions:


resource "google_project_iam_member" "service_account" {
  for_each = toset([
    "roles/compute.viewer",
    "roles/cloudasset.viewer",
    "roles/monitoring.viewer",
  ])

  project = var.project
  role    = each.value
  member  = "serviceAccount:${google_service_account.datadog.email}"
}

chruzcruz commented 1 year ago

Same issue here on 3.24.0

resource "google_project_service" "enable_apis" {
  for_each = toset([
    "pubsub.googleapis.com",
    "cloudbilling.googleapis.com",
    "monitoring.googleapis.com",
    "compute.googleapis.com",
    "cloudasset.googleapis.com",
  ])
  project            = local.project_id
  service            = each.value
  disable_on_destroy = false

  resource "google_service_account" "datadog" {
  project = local.project_id
  account_id   = "datadog-integration-account"
  display_name = "Datadog Integration with ${local.project_id}"
}

resource "google_service_account_key" "datadog" {
  service_account_id = google_service_account.datadog.name
}

  resource "google_project_iam_member" "service_account" {
  for_each = toset([
    "roles/compute.viewer",
    "roles/container.viewer",
    "roles/monitoring.viewer",
    "roles/cloudasset.viewer",
  ])

  project = local.project_id
  role    = each.value
  member  = "serviceAccount:${google_service_account.datadog.email}"
}

}

resource "datadog_integration_gcp" "dd_gcp_project_integration" {
  project_id     = jsondecode(base64decode(google_service_account_key.datadog.private_key))["project_id"]
  private_key    = jsondecode(base64decode(google_service_account_key.datadog.private_key))["private_key"]
  private_key_id = jsondecode(base64decode(google_service_account_key.datadog.private_key))["private_key_id"]
  client_email   = jsondecode(base64decode(google_service_account_key.datadog.private_key))["client_email"]
  client_id      = jsondecode(base64decode(google_service_account_key.datadog.private_key))["client_id"]
  host_filters   = join(",", sort(var.host_filters))

}

Output

- Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions

nkzou commented 1 year ago

The cause of this issue is not due to the Terraform Provider, but rather some underlying configuration error with the Datadog integration and/or GCP. That specific error message is a catch-all, with a variety of possible misconfigurations - we cannot help debug this in a Github issue. Please open a ticket with Datadog support at https://help.datadoghq.com/, and a support engineer will work with you to resolve the issue with much more visibility than we can here.

DataDog / terraform-provider-datadog