Cannot import `google_billing_budget`

pikesley commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

# terraform -v
Terraform v0.12.24
+ provider.google v3.13.0
+ provider.google-beta v3.15.0

Affected Resource(s)

google_billing_budget

Terraform Configuration Files

resource google_billing_budget default {
  provider        = google-beta
  billing_account = "AAAAAA-000000-EEEEEE"
}

Debug Output

Expected Behavior

We have an existing Billing Budget and we expected to be able to import it into Terraform (using these instructions)

Actual Behavior

The import fails with in a confusing way.

Steps to Reproduce

I configured the resource as shown above (with a legit billing_account id) and then:

# terraform import -provider=google-beta google_billing_budget.default billingAccounts/AAAAAA-000000-EEEEEE/budgets/uuid-for-the-budget
google_billing_budget.default: Importing from ID "billingAccounts/AAAAAA-000000-EEEEEE/budgets/uuid-for-the-budget"...

Error: project: required field is not set

which feels wrong - why does a Billing Budget need a project?

Anyway, I then tried setting the GOOGLE_PROJECT var, but I found that I could set it to any string - Terraform does not seem to care as long as it exists:

# GOOGLE_PROJECT=foo terraform import -provider=google-beta google_billing_budget.default billingAccounts/AAAAAA-000000-EEEEEE/budgets/uuid-for-the-budget
google_billing_budget.default: Importing from ID "billingAccounts/AAAAAA-000000-EEEEEE/budgets/uuid-for-the-budget"...
google_billing_budget.default: Import prepared!
  Prepared google_billing_budget for import
google_billing_budget.default: Refreshing state... [id=billingAccounts/AAAAAA-000000-EEEEEE/budgets/uuid-for-the-budget]

Error: Error reading BillingBudget "billingAccounts/AAAAAA-000000-EEEEEE/budgets/uuid-for-the-budget": googleapi: Error 403: Cloud Billing Budget API has not been used in project 123412341234 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/billingbudgets.googleapis.com/overview?project=123412341234 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

The actual project ID it threw up (which I've replaced there with 123412341234) is the ID for our terraform-admin project, but this choice seems to be entirely arbitrary - after all, I set GOOGLE_PROJECT=foo

Important Factoids

I tried changing the underlying Gcloud context with gcloud config set project to see if this made any difference, but got the same result.

This may be down to us misunderstanding how this is supposed to work, but the fact it requires GOOGLE_PROJECT to be set, but then doesn't care what it's set to, definitely smells like a bug.

b/301412737

venkykuberan commented 4 years ago

@pikesley couple of things going on here, let me try to break it down

1) Terraform provider needs the Project variable to be set in one of the ways (env. variable, provider config, resource config) in order to run. Although this billing API call doesn't require the project id, the validation happens before the API call to check project id value is set. Project id is validated once API call is made.

2) Cloud console only shows the displayName attribute on the UI. However in order to export the billing_budget resource we need the name attribute. To get the actual name of the billing budget resource please use the following api then use the import command. Please let me know if it helps.

terraform import -provider=google-beta google_billing_budget.default {{name}}

API - https://cloud.google.com/billing/docs/reference/budget/rest/v1beta1/billingAccounts.budgets/list

Sample Response -

{
  "budgets": [
    {
      "name": "billingAccounts/01xxx-8xxx-xxxx/budgets/xxxxxx",
      "displayName": "myBudget",
      "budgetFilter": {
        "projects": [
          "projects/xxxx",
          "projects/xxxx"
        ],
        "creditTypesTreatment": "INCLUDE_ALL_CREDITS"
      },
      "amount": {
        "specifiedAmount": {
          "currencyCode": "USD",
          "units": "1000"
        }
      },
      "thresholdRules": [
        {
          "thresholdPercent": 0.5,
          "spendBasis": "CURRENT_SPEND"
        },
        {
          "thresholdPercent": 1,
          "spendBasis": "CURRENT_SPEND"
        },
        {
          "thresholdPercent": 0.9,
          "spendBasis": "FORECASTED_SPEND"
        }
      ],
      "allUpdatesRule": {},
      "etag": "7eb9ad05e3b62f43"
    }

zkauker commented 4 years ago

@venkykuberan could we warm up this topic a little bit? In general there are multiple problems around importing (or just reading) budget resources:

Before the resource import there is an option to tokenize the import ID and extract information from that. This logic fails, so the import fails as well
After fixing this problem it turns out that the API changed because the details of the budget in the response is not under the budget field anymore, so while the import technically succeeds, none of the budget attributes gets mapped into the terraform resource in the state file - so you won't be able to get a clean state
The budget info coming from the API does not provide the billing_account attribute, however, this is a required resource attribute in terraform. In order to make it possible to get a clean state, I think the resource read logic should somehow map the billing account ID (contained by the resource ID) into the terraform resource's billing_account attribute.

I was dumb enough not to recognize that the provider code is mostly generated, so I issued a PR last night for the .go code that fixes these issues. Obviously, it cannot be merged, but it might be helpful for you to understand the problems around budgets, you can find it here: https://github.com/hashicorp/terraform-provider-google-beta/pull/2660

Let me know what do you think!

davideasaf commented 2 years ago

Similar issue for google_bigquery_data_transfer_config resource

https://github.com/hashicorp/terraform-provider-google/issues/10788#issuecomment-1059394979

hashicorp / terraform-provider-google