Enable Required Google APIs as a terraform resource

This is a simple proposal but it would be nice if the prerequisites for setting up the Autoscaler could be handled within the terraform file itself:

gcloud services enable iam.googleapis.com \
  cloudresourcemanager.googleapis.com \
  appengine.googleapis.com \
  firestore.googleapis.com \
  spanner.googleapis.com \
  pubsub.googleapis.com \
  logging.googleapis.com \
  monitoring.googleapis.com \
  cloudfunctions.googleapis.com \
  cloudbuild.googleapis.com \
  cloudscheduler.googleapis.com \
  cloudresourcemanager.googleapis.com

becomes

resource "google_project_service" "project" {
  for_each = toset(["appengine.googleapis.com",
    "artifactregistry.googleapis.com",
    "firestore.googleapis.com",
    "spanner.googleapis.com",
    "pubsub.googleapis.com",
    "cloudfunctions.googleapis.com",
    "cloudbuild.googleapis.com",
    "cloudscheduler.googleapis.com",
  "cloudresourcemanager.googleapis.com"])

  service = each.key
  disable_on_destroy = false
}

Potentially we would need to add a delay in terraform after running the create on this resource

Hi @klanmiko, thanks for raising the issue, this is a good suggestion and one that we have experimented with. As you noted, there are sometimes timing challenges when enabling APIs with Terraform that can result in the need to introduce complex dependencies or to introduce artificial delays to ensure service availability when required.

As well as this, as part of the deployment of the Autoscaler in certain configurations, it's required to enable the App Engine API upfront (for Cloud Scheduler/Firestore), update Firestore to native mode, and import the app into Terraform state -- unfortunately it's not possible to handle all App Engine management via Terraform because it would cause terraform destroy to fail, see the note here:

"App Engine applications cannot be deleted once they're created; you have to delete the entire project to delete the application. Terraform will report the application has been successfully deleted; this is a limitation of Terraform, and will go away in the future. Terraform is not able to delete App Engine applications."

The GKE module additionally requires early read access (i.e. during the plan stage) to the Compute and GKE APIs to retrieve zonal and GKE version information. This means that an upfront gcloud services enable call would be required anyway.

Because of the above, for consistency we chose to retain the current model of enabling all required APIs upfront in parallel (with a few small conditional exceptions). I'll close this issue, but please feel free to reopen if you'd like to continue the discussion, and thanks again for raising this.

cloudspannerecosystem / autoscaler

Enable Required Google APIs as a terraform resource #358