google_billing_budget: Error creating Budget: googleapi: Error 400: Request contains an invalid argument.

n-yassine commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

0.13.7

Affected Resource(s)

google_billing_budget

Terraform Configuration Files

I followed the documentation (here) with the basic format of the resource google_billing_budget


provider "google" {
  region  = "europe-west1" 
  version = "~> 3.72.0" 
}

provider "google-beta" {
  region  = "europe-west1"
  version = "~> 3.72.0"
}

resource "google_billing_budget" "budget" {
  billing_account = BILLING_ACCOUNT_ID
  display_name = "Example Billing Budget"
  amount {
    specified_amount {
      currency_code = "USD"
      units = "100000"
    }
  }
  threshold_rules {
      threshold_percent =  0.5
  }
}

And I got the following error

Debug Output

---[ REQUEST ]---------------------------------------
POST /v1/billingAccounts/BILLING_ACCOUNT_ID/budgets?alt=json HTTP/1.1
Host: billingbudgets.googleapis.com
User-Agent: Terraform/0.13.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/3.72.0
Content-Length: 183
Content-Type: application/json
Accept-Encoding: gzip
{
 "amount": {
  "specifiedAmount": {
   "currencyCode": "USD",
   "units": "100000"
  }
 },
 "displayName": "Example Billing Budget",
 "thresholdRules": [
  {
   "spendBasis": "CURRENT_SPEND",
   "thresholdPercent": 0.5
  }
 ]
}
-----------------------------------------------------: timestamp=2021-06-15T15:17:00.449Z
2021-06-15T15:17:00.527Z [INFO]  plugin.terraform-provider-google-beta_v3.72.0_x5: 2021/06/15 15:17:00 [INFO] error retrieving userinfo for your provider credentials. have you enabled the 'https://www.googleapis.com/auth/userinfo.email' scope? error: error retrieving userinfo for your provider credentials. have you enabled the 'https://www.googleapis.com/auth/userinfo.email' scope? error: googleapi: got HTTP response code 401 with body: {
  "error": "invalid_request",
  "error_description": "Invalid Credentials"
}: timestamp=2021-06-15T15:17:00.526Z
2021-06-15T15:17:00.527Z [INFO]  plugin.terraform-provider-google-beta_v3.72.0_x5: 2021/06/15 15:17:00 [INFO] Terraform is using this identity:: timestamp=2021-06-15T15:17:00.526Z
2021-06-15T15:17:00.533Z [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021-06-15T15:17:00.537Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/hashicorp/google-beta/3.72.0/linux_amd64/terraform-provider-google-beta_v3.72.0_x5 pid=332
2021-06-15T15:17:00.537Z [DEBUG] plugin: plugin exited
Warning: Deprecated Resource
The null_data_source was historically used to construct intermediate values to
re-use elsewhere in configuration, the same can now be achieved using locals
Error: Error creating Budget: googleapi: Error 400: Request contains an invalid argument.
  on billing_alert.tf line 5, in resource "google_billing_budget" "budget":
   5: resource "google_billing_budget" "budget" {
2021/06/15 15:17:01 [DEBUG] google_billing_budget.budget: apply errored, but we're indicating that via the Error pointer rather than returning it: Error creating Budget: googleapi: Error 400: Request contains an invalid argument.
2021/06/15 15:17:01 [ERROR] eval: *terraform.EvalApplyPost, err: Error creating Budget: googleapi: Error 400: Request contains an invalid argument.
2021/06/15 15:17:01 [ERROR] eval: *terraform.EvalSequence, err: Error creating Budget: googleapi: Error 400: Request contains an invalid argument.
2021-06-15T15:17:01.365Z [INFO]  plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:17:01 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 400 Bad Request
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Tue, 15 Jun 2021 15:17:01 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0
{
  "error": {
    "code": 400,
    "message": "Request contains an invalid argument.",
    "status": "INVALID_ARGUMENT"
  }
}
-----------------------------------------------------: timestamp=2021-06-15T15:17:01.360Z
2021-06-15T15:17:01.365Z [INFO]  plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:17:01 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 400 with body: HTTP/2.0 400 Bad Request
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Tue, 15 Jun 2021 15:17:01 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0
{
  "error": {
    "code": 400,
    "message": "Request contains an invalid argument.",
    "status": "INVALID_ARGUMENT"
  }
}: timestamp=2021-06-15T15:17:01.360Z
360Z
2021-06-15T15:17:01.523Z [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021-06-15T15:17:01.523Z [DEBUG] plugin: plugin process exited: path=/usr/bin/terraform pid=295
2021-06-15T15:17:01.523Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/hashicorp/google/3.72.0/linux_amd64/terraform-provider-google_v3.72.0_x5 pid=313
2021-06-15T15:17:01.523Z [DEBUG] plugin: plugin exited
2021-06-15T15:17:01.523Z [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021-06-15T15:17:01.524Z [DEBUG] plugin: plugin exited

Panic Output

Expected Behavior

Actual Behavior

Error: Error creating Budget: googleapi: Error 400: Request contains an invalid argument.
  on billing_alert.tf line 5, in resource "google_billing_budget" "budget":
   5: resource "google_billing_budget" "budget" {

Steps to Reproduce

terraform apply

Important Factoids

For information, the service account used to create the resources has a scope: cloud-platform And this role: roles/billing.admin

I have the impression that the problem is at the send notification step.

2021-06-15T15:17:00.527Z [INFO]  plugin.terraform-provider-google-beta_v3.72.0_x5: 2021/06/15 15:17:00 [INFO] error retrieving userinfo for your provider credentials. have you enabled the 'https://www.googleapis.com/auth/userinfo.email' scope? error: error retrieving userinfo for your provider credentials. have you enabled the 'https://www.googleapis.com/auth/userinfo.email' scope? error: googleapi: got HTTP response code 401 with body: {
  "error": "invalid_request",
  "error_description": "Invalid Credentials"
}: timestamp=2021-06-15T15:17:00.526Z

Below are the activated APIs

"storage-api.googleapis.com",
  "compute.googleapis.com",
  "oslogin.googleapis.com",
  "cloudresourcemanager.googleapis.com",
  "cloudbilling.googleapis.com",
  "serviceusage.googleapis.com",
  "iam.googleapis.com",
  "iamcredentials.googleapis.com",
  "pubsub.googleapis.com",                  
  "dns.googleapis.com",                      
  "containerregistry.googleapis.com",
  "containeranalysis.googleapis.com",
  "containerscanning.googleapis.com",
  "appengine.googleapis.com",
  "logging.googleapis.com",    
  "monitoring.googleapis.com",
  "billingbudgets.googleapis.com",

References

0000

venkykuberan commented 3 years ago

Looks like terraform is not setup with any creds., Please follow this page https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/getting_started#adding-credentials

mldmld68 commented 3 years ago

Hi

The job is run by a Gitlab runner GCE running docker image. The GCE has a attached service account which has cloud-platform scope.

Earlier in the log, one may read information showing Terraform has credential because it can get access to the bucket of the tfstate and some info (I anonymized) about the GCP project

Regards MLD

-----------------------------------------------------: timestamp=2021-06-15T15:16:41.352Z 2021-06-15T15:16:41.353Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [DEBUG] Retry Transport: Stopping retries, last request was successful: timestamp=2021-06-15T15:16:41.352Z 2021-06-15T15:16:41.353Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [DEBUG] Retry Transport: Returning after 1 attempts: timestamp=2021-06-15T15:16:41.352Z 2021-06-15T15:16:41.354Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [DEBUG] Read bucket xxxxxxxxxxxxxxxxxxxxxxxxx at location https://www.googleapis.com/storage/v1/b/xxxxxxxxxxxxxxxxxxxxxxxxx: timestamp=2021-06-15T15:16:41.352Z 2021-06-15T15:16:41.355Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [INFO] Instantiating Google Cloud ResourceManager client for path https://cloudresourcemanager.googleapis.com/: timestamp=2021-06-15T15:16:41.355Z 2021-06-15T15:16:41.355Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [DEBUG] Retry Transport: starting RoundTrip retry loop: timestamp=2021-06-15T15:16:41.355Z 2021-06-15T15:16:41.355Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [DEBUG] Retry Transport: request attempt 0: timestamp=2021-06-15T15:16:41.355Z 2021-06-15T15:16:41.356Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [DEBUG] Google API Request Details: ---[ REQUEST ]--------------------------------------- GET /v1/projects/xxxxxxxxxxxxxxxxxxxxxxxxx?alt=json&prettyPrint=false HTTP/1.1 Host: cloudresourcemanager.googleapis.com User-Agent: google-api-go-client/0.5 Terraform/0.13.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/3.72.0 X-Goog-Api-Client: gl-go/1.16.2 gdcl/20210502 Accept-Encoding: gzip

-----------------------------------------------------: timestamp=2021-06-15T15:16:41.355Z [0m[1mgoogle_project_service.main[2]: Refreshing state... [id=xxxxxxxxxxxxx/audit.googleapis.com][0m 2021-06-15T15:16:41.460Z [INFO] plugin.terraform-provider-google_v3.72.0_x5: 2021/06/15 15:16:41 [DEBUG] Google API Response Details: ---[ RESPONSE ]-------------------------------------- HTTP/2.0 200 OK Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43" Cache-Control: private Content-Type: application/json; charset=UTF-8 Date: Tue, 15 Jun 2021 15:16:41 GMT Server: ESF Server-Timing: gfet4t7; dur=126 Vary: Origin Vary: X-Origin Vary: Referer X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-Xss-Protection: 0

{ "projectNumber": "----------8151", "projectId": "xxxxxxxxx", "lifecycleState": "ACTIVE", "name": "xxxxxxxxxxxx", "labels": { "buyer": "xxxxx", ....

slevenick commented 3 years ago

Huh, that's a pretty unhelpful error...

I can ask around, but that appears to be a fine request, and it's based off our basic test that seems to pass every day.

For instance, our CI ran the test last night and passed after sending this as the request body:

{
 "amount": {
  "specifiedAmount": {
   "currencyCode": "USD",
   "units": "100000"
  }
 },
 "displayName": "Example Billing Budget3am0vbuwut",
 "thresholdRules": [
  {
   "spendBasis": "CURRENT_SPEND",
   "thresholdPercent": 0.5
  }
 ]
}

What format is your billing budget ID in? It should look similar to: 123A45-678901-B23C45

slevenick commented 3 years ago

I would definitely be suspicious of that 401 authentication issue. Is it possible that the account you are running Terraform on has access to the storage bucket/project but not the billing account? Billing accounts generally have stricter permissions that are controlled through the billing account/organization itself rather than at the project level

n-yassine commented 3 years ago

Hi slevenick,

terraform plan give me this result:

Terraform will perform the following actions:
  # google_billing_budget.budget will be created
  + resource "google_billing_budget" "budget" {
      + billing_account = "01D737-CD1044-XXXXXX"
      + display_name    = "Example Billing Budget"
      + id              = (known after apply)
      + name            = (known after apply)
      + amount {
          + specified_amount {
              + currency_code = "USD"
              + units         = "100000"
            }
        }
      + budget_filter {
          + credit_types           = (known after apply)
          + credit_types_treatment = "EXCLUDE_ALL_CREDITS"
          + labels                 = (known after apply)
          + projects               = [
              + "projects/56756620XXXX",
            ]
          + services               = []
          + subaccounts            = (known after apply)
        }
      + threshold_rules {
          + spend_basis       = "CURRENT_SPEND"
          + threshold_percent = 0.5
        }
    }

But with: terraform apply I get the error mentioned above

I have checked that the service account used, it has the role roles/billing.admin at the organization level to create alerts

I remain at your disposal for any further information.

Yassine

slevenick commented 3 years ago

Nevermind, disregard

n-yassine commented 3 years ago

Hi,

The problem has been solved. The error reported above has no link with the source of the issue.

The issue was in the currency_code parameter of the billing alerts configuration.

In the configuration we have to use the same currency code as the one used by the billing account. The conversion is not done automatically.

In our case, the currency of the billing account is EUR but in the configuration we put USD.

...
 "amount": {
  "specifiedAmount": {
   "currencyCode": "EUR"
   "units": "10000"
  }

I think that the error message generated by this misconfiguration should be reviewed to make it more explicit

I hope this can help other people.

github-actions[bot] commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-google