Error enabling APIs on GCP project

ank9it commented 5 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Terraform v0.11.11

Affected Resource(s)

google_project_services

Terraform Configuration Files

resource "google_project" "runtime_project" {
 name            = "${var.projname}"
 project_id      = "${var.projname}-${random_integer.project_suffix.result}"
 billing_account = "${var.billing_account}"
 org_id          = "${var.org_id}"
 auto_create_network = false
}

resource "google_project_services" "runtime_project" {
 project = "${google_project.runtime_project.project_id}"
 services = [
   "compute.googleapis.com",
   "dns.googleapis.com",
   "sql-component.googleapis.com",
   "sqladmin.googleapis.com",
   "cloudresourcemanager.googleapis.com",
   "deploymentmanager.googleapis.com",
   "dataproc.googleapis.com",
   "iam.googleapis.com",
   "stackdriver.googleapis.com"
 ]

 depends_on = ["google_project.runtime_project"]

}

Expected Behavior

Should activate the APIs

Actual Behavior

google_project_services.runtime_project: Error creating services: Error waiting for apis ["cloudresourcemanager.googleapis.com" "dataproc.googleapis.com" "deploymentmanager.googleapis.com" "dns.googleapis.com" "iam.googleapis.com" "sql-component.googleapis.com" "sqladmin.googleapis.com" "stackdriver.googleapis.com"] to be enabled for ak09-97261: Error code 9, message: The service is currently being deactivated and deactivation must complete before activation can occur.:[]

Steps to Reproduce

terraform apply

rileykarson commented 5 years ago

Hey @ank9it!

Are you experiencing this when running as a single terraform apply creating the new project? And can you include debug logs?

ank9it commented 5 years ago

Hello @rileykarson,

Just noticed that this issue only occurs only if I use auto_create_network = false. Here's the debug logs.

v6 commented 5 years ago

I'm running into this too:

Error: Error applying plan:

2 error(s) occurred:

* google_project_service.serviceusage-api: 1 error(s) occurred:

* google_project_service.serviceusage-api: Error enabling service: failed to issue request: googleapi: Error 403: Service Usage API has not been used in project 166274966491 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=166274966491 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry., accessNotConfigured
* google_project_service.servicenetworking-api: 1 error(s) occurred:

• google_project_service.servicenetworking-api: Error enabling service: failed to issue request: googleapi: Error 403: Service Usage API has not been used in project 166274966491 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=166274966491 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry., accessNotConfigured

Seems kind of like a catch 22. The error says I can't enable it because it isn't enabled?

v6 commented 5 years ago

In the mean time, what can I do to work around this?

Null resource?

rileykarson commented 5 years ago

So, the issue impacting you ends up being pretty easy to fix- it's a slightly incorrect config. It's not incorrect for a good reason, and I'll get into the details further below.

If you add oslogin.googleapis.com to the list of services, this should work as intended. It's a dependency of compute.googleapis.com, and if you look at this line in the logs:

2019/03/07 02:48:42 [DEBUG]: Disabling oslogin.googleapis.com as it's enabled upstream but not in config

When we disable it, we [necessarily] disable the compute API. Later on, we attempt to enable both again and they haven't finished disabling.

Unless having other services disabled is very important to you, I would recommend using https://www.terraform.io/docs/providers/google/r/google_project_service.html.

The larger issue for Terraform is that we're unable to tell before apply time if we've specified a correct set of services, or communicate that an incorrect set was chosen to the user. This is somewhere between working as intended / infeasible to provide a good solution to right now.

First, we can test enabling compute.googleapis.com in a fresh project. As expected, oslogin.googleapis.com is enabled as well- the oslogin service is a dependency of the compute service.

Since your list doesn't include oslogin, Terraform correctly disables it, and that includes disabling the compute service as a dependent. The API doesn't provide a batch disable endpoint, so we need to disable services one by one. Terraform can't order the disablement of services by dependency graph (none is published anywhere) that means we need to force disable dependents. This is for cases where we disable a service (like compute) before the dependency (like oslogin).

We need to hope we don't enter an invalid state where we've disabled a service we actually want enabled. ("Hope" sounds pretty dire- if the correct set of services is specified, as enabled, we'll never enter an invalid state. Terraform isn't able to tell ahead of time if it will enter one w/o knowing the dependency graph, though.)

When we disable oslogin + compute as a dependent, the API reports that disablement is finished. That's incorrect, and they aren't done being disabled, based on the error message we've received.

Finally, we start enabling the services that were not previously enabled. We won't enable compute.googleapis.com here even though it's been disabled- Terraform doesn't catch changes like this mid-apply. We will enable dataproc.googleapis.com, which definitely depends on compute. So enabling dataproc will enable compute and oslogin, both of which aren't done being disabled. That causes the API to send an error, which is the message you've received:

The service is currently being deactivated and deactivation must complete before activation can occur.:[]

It's also a problem that we're getting an empty list back. I'll file an issue against the team responsible for this API for it responding with that unhelpful of an error message.

@v6: I'm not sure that's related. Can you repro + provide debug logs in a separate issue?

femrtnz commented 5 years ago

I see a similar issue when create the project and try to enable cloudbilling.googleapis.com

I can see the terraform result as successful, but the api still keep disabled

resource "google_project" "project" {
  name                = "abc"
  project_id          = "abc-abc"
  auto_create_network = false
  skip_delete         = true
  billing_account     = "${var.billing_id}"
}

resource "google_project_services" "services" {
  project                    = "${google_project.project.project_id}"
  services                   = ["cloudbilling.googleapis.com"]
}

rileykarson commented 5 years ago

Hey @femrtnz!

Do you mind sharing debug logs?

femrtnz commented 5 years ago

Hey @rileykarson Please ignore my message as I can't reproduce the error anymore.

rileykarson commented 5 years ago

Thanks @femrtnz!

I'm going to close this issue out- I'm not aware of any (unexpected) active issues, and there isn't really anything here for a maintainer to take action on. If anyone encounters this or a similar issue, please file a new issue (ideally with debug logs) so we can take a look.

sarab321 commented 5 years ago

Hi,

I am still facing this issue. What was the resolution and how you resolve it

ghost commented 5 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

hashicorp / terraform-provider-google