Adding support for specify 'X-Goog-User-Project' HTTP header for quota and billing purposes for budget

[laithrafid]

i'm getting error below when using budget sub-module when i use access token generated from gcloud auth print-access-token , you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.

 Error: Error creating Budget: googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the billingbudgets.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/. If you are getting this error with curl or similar tools, you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/32555940559",
│       "service": "billingbudgets.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│
│   with module.budget_project_factory.google_billing_budget.budget[0],
│   on .terraform/modules/budget_project_factory/modules/budget/main.tf line 38, in resource "google_billing_budget" "budget":
│   38: resource "google_billing_budget" "budget" {
│

[morgante]

You should set the billing_project configuration on your provider config. For example:

provider "google" {
  billing_project = "your-billing-project"
}
provider "google-beta" {
  billing_project = "your-billing-project"
}

[laithrafid]

putting billing project before creating it ? it doesn't make sense https://github.com/laithrafid/infra-api/blob/main/terraform/googlecloud/modules/gke_create/main.tf using it to make a budget and alert for a project after creating the project

[morgante]

You need to choose an "admin" or "bootstrap" project that manages the rest of your configurations, including creating new projects. That project can be used as the billing_project.

It doesn't need to be the same project you're actually creating.

[laithrafid]

yes that another approach , however i don't like to create bootstrap/admin project then create a project that i'm really using , this's not and shouldn't be a best practice. , however this error is related to Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the billingbudgets.googleapis.com. this module backend project-factory (google/google-beta provider) not supporting the use of access token api calls to billingbudgets.googleapis.com service without passing 'X-Goog-User-Project'.

[morgante]

Google APIs require that they be activated on the project your credentials are coming from. For convenience, Google provides a default project (for the Cloud Shell) that has some APIs activated, but the billingbudgets API is not active on that project. Thus, you must specify a project of your own to use for API activation.

One potential workaround could be to have an alternative provider configuration specifically for the billing module:

provider "google" {
  alias = "project"
  billing_project = "your-billing-project"
}

module "budget_project_factory" {
  source                 = "terraform-google-modules/project-factory/google//modules/budget"
  providers = {
    google = google.project
  }
}

[laithrafid]

billingbudgets.googleapis.com enabled and working on newly created project , but still face same issue/error ... even with your workaround still asks for var.activate_apis :

activate_apis = [
  "compute.googleapis.com",
  "container.googleapis.com",
  "servicemanagement.googleapis.com",
  "servicecontrol.googleapis.com",
  "endpoints.googleapis.com",
  "serviceusage.googleapis.com",
  "servicenetworking.googleapis.com",
  "logging.googleapis.com",
  "bigquery.googleapis.com",
  "cloudresourcemanager.googleapis.com",
  "billingbudgets.googleapis.com",
  "cloudbilling.googleapis.com",
  "iam.googleapis.com",
  "admin.googleapis.com",
  "appengine.googleapis.com",
  "storage-api.googleapis.com",
  "monitoring.googleapis.com"
]

module "project_factory" {
  source          = "terraform-google-modules/project-factory/google"
  version         = ">= 12.0.0"
  name            = var.project_name
  project_id      = local.project_id
  org_id          = var.organization_id
  billing_account = var.billing_account
  folder_id       = google_folder.folder.id
  activate_apis   = var.activate_apis
  create_project_sa = false 
  lien              = var.lien
  depends_on = [
    google_folder.folder
  ]
}
module "quota_project_factory" {
  source          = "terraform-google-modules/project-factory/google//modules/quota_manager"
  project_id      = module.project_factory.project_id
  consumer_quotas = var.consumer_quotas
}

resource "google_pubsub_topic" "budget" {
  name    = "budget-topic-${module.project_factory.project_id}"
  project = module.project_factory.project_id
}

module "budget_project_factory" {
  source                 = "terraform-google-modules/project-factory/google//modules/budget"
  version                = ">= 12.0.0"
  providers = {
    google = google.project
  }
  billing_account        = var.billing_account
  projects               = [module.project_factory.project_id]
  amount                 = var.budget_amount
  credit_types_treatment = var.budget_credit_types_treatment
  services               = var.budget_services
  alert_spent_percents   = var.budget_alert_spent_percents
  alert_pubsub_topic     = "projects/${module.project_factory.project_id}/topics/${google_pubsub_topic.budget.name}"
  labels = {
    "Enviornment" : "dev"
  }
  depends_on = [
    google_pubsub_topic.budget,
    module.project_factory
  ]
}

providers.tf

terraform {
  cloud {
    organization = "bayt"
    hostname     = "app.terraform.io"
    workspaces {
      name = "intra-api-googlecloud"
      //tags = ["APIs:digitalocean"]
    }
  }
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 4.14.0"
    }
    gsuite = {
      source  = "hashicorp/googleworkspace"
      version = ">= 0.6.0"
    }
  }
}

provider "google" {
  project      = var.project_name
  region       = var.region
  access_token = var.GOOGLECLOUD_TOKEN
  # credentials  = var.credentials
  scopes = [
    # Default scopes
    "https://www.googleapis.com/auth/compute",
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/ndev.clouddns.readwrite",
    "https://www.googleapis.com/auth/devstorage.full_control",

    # Required for google_client_openid_userinfo
    "https://www.googleapis.com/auth/userinfo.email",
  ]
}
provider "google" {
  alias = "project"
  access_token = var.GOOGLECLOUD_TOKEN
  billing_project = var.project_name
}
provider "google-beta" {
  project      = var.project_name
  region       = var.region
  access_token = var.GOOGLECLOUD_TOKEN
  # credentials  = var.credentials

  scopes = [
    # Default scopes
    "https://www.googleapis.com/auth/compute",
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/ndev.clouddns.readwrite",
    "https://www.googleapis.com/auth/devstorage.full_control",

    # Required for google_client_openid_userinfo
    "https://www.googleapis.com/auth/userinfo.email",
  ]
}

Same Error

Error: Error creating Budget: googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the billingbudgets.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/. If you are getting this error with curl or similar tools, you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/32555940559",
│       "service": "billingbudgets.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }

[morgante]

Try this:

provider "google" {
  alias = "project"
  access_token = var.GOOGLECLOUD_TOKEN
  billing_project = var.project_name
  user_project_override = true
}

[laithrafid]

No it doesn't work , this's a bug in provider code and should be fixed

[mehdicopter]

how did you manage to fix ti @laithrafid ?