Closed Stono closed 4 years ago
OK so this might be a google side problem, as you can see, bigquery isn't even being returned from the gcloud cli:
β― gcloud services list --project=at-delivery-platform-testing
NAME TITLE
bigquery-json.googleapis.com BigQuery API
bigquerystorage.googleapis.com BigQuery Storage API
cloud-file-service-gcp.elastifile.com Elastifile Cloud File Service
cloudfunctions.googleapis.com Cloud Functions API
cloudkms.googleapis.com Cloud Key Management Service (KMS) API
cloudresourcemanager.googleapis.com Cloud Resource Manager API
cloudtrace.googleapis.com Stackdriver Trace API
compute.googleapis.com Compute Engine API
container.googleapis.com Kubernetes Engine API
containerregistry.googleapis.com Container Registry API
dataflow.googleapis.com Dataflow API
deploymentmanager.googleapis.com Cloud Deployment Manager V2 API
dialogflow.googleapis.com Dialogflow API
distance-matrix-backend.googleapis.com Distance Matrix API
dns.googleapis.com Google Cloud DNS API
firebaserules.googleapis.com Firebase Rules API
firestore.googleapis.com Cloud Firestore API
geocoding-backend.googleapis.com Geocoding API
geolocation.googleapis.com Geolocation API
iam.googleapis.com Identity and Access Management (IAM) API
iamcredentials.googleapis.com IAM Service Account Credentials API
logging.googleapis.com Stackdriver Logging API
monitoring.googleapis.com Stackdriver Monitoring API
oslogin.googleapis.com Cloud OS Login API
pubsub.googleapis.com Cloud Pub/Sub API
redis.googleapis.com Google Cloud Memorystore for Redis API
replicapool.googleapis.com Compute Engine Instance Group Manager API
replicapoolupdater.googleapis.com Compute Engine Instance Group Updater API
resourceviews.googleapis.com Compute Engine Instance Groups API
servicemanagement.googleapis.com Service Management API
servicenetworking.googleapis.com Service Networking API
serviceusage.googleapis.com Service Usage API
source.googleapis.com Legacy Cloud Source Repositories API
sqladmin.googleapis.com Cloud SQL Admin API
stackdriver.googleapis.com Stackdriver API
stackdriverprovisioning.googleapis.com Stackdriver Provisioning Service
storage-api.googleapis.com Google Cloud Storage JSON API
storage-component.googleapis.com Cloud Storage
websecurityscanner.googleapis.com Web Security Scanner API
Hey @Stono! This happened because of some changes upstream that made bigquery.googleapis.com
the new name for bigquery-json.googleapis.com
. It's been since rolled back when it caused issues for Terraform users, and we're working with that team to mitigate the impact on Terraform users before they roll the change forward again.
I'm going to use this issue to track that. In the meantime, if you've defined bigquery.googleapis.com
in config, you can remove it. bigquery-json.googleapis.com
remains the service's canonical name for now. If you're encountering this issue and that value isn't in your config, please let me know.
Thanks a lot @rileykarson - I've removed bigquery.googleapis.com
and things seem OK.
We've had a right nightmare because of this, and accidentally disabled bigquery api on some of our projects as a result when the upstream change was made! Do you have any more details with regards to that change or where Google announced the rename so I can use it in a post mortem?
thanks @rileykarson, this caused us issues as well - originally, and now again when it was rolled back :). I was wondering if there are any release notes or more details about this change available anywhere? (I was not able to find any mention)
I'm not aware of any release notes being published- I'll update this issue if I'm made aware of any. I'll also update the TPG/TPGB changelogs to note that users may have encountered a disruption.
This should be mitigated in release 2.17.0
onwards. Specifying either service (bigquery-json.googleapis.com
or bigquery.googleapis.com
) in google_project_service
and google_project_services
should work (or both services for _services
).
Effectively, Terraform will treat the values as equivalent. It'll retry failed requests using the alternate value in case one of the values no longer works. If there's a real reason for failure and both fail, it'll return the errors.
Upon reading either value from the API, Terraform will record what you've defined in your config in state. If you've defined neither, Terraform will write the bigquery-json.googleapis.com
value. Transitioning between the values will show a diff, but Terraform should take no action. Once you apply the update, Terraform will have the correct value recorded in state.
Additionally, I'm going to disallow the bigquery-json.googleapis.com
service in google_project_service
from 3.0.0
on https://github.com/terraform-providers/terraform-provider-google/issues/4624.
I'm going to lock this issue because it has been closed for 30 days β³. This helps our maintainers find and focus on the active issues.
If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error π€ π , please reach out to my human friends π hashibot-feedback@hashicorp.com. Thanks!
Community Note
Hi, We are seeing an issue where terraform plans are always trying to enable the biquery api on our project, even though it is already enabled. Even if we push through to an apply, it never completes (with the errors seen below).
Terraform Version
terraform: 0.12.9, provider: 2.16.0
Affected Resource(s)
google_project_services
Terraform Configuration Files
Debug Output
We see:
And then we just see it polling:
We can see the action was performed in the GCP Audit:
And the BigQuery API is enabled
I find interesting in the debug output too its asking for a list of enabled api's and that doesn't seem to include bigquery (probably explains the reason it's showing in the plan):
Panic Output
Expected Behavior
Terraform should manage the apis deterministically
Actual Behavior
We're unable to complete a terraform apply as the api modification times out after 20 minutes with the following error:
Steps to Reproduce
terraform apply
Important Factoids
References
0000