Terraform is unable to access cloudbilling.googleapis.com when running the code in GKE pod/GCE VM

[ghost]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.1.9 on linux_amd64 google provider version 4.9.0

Affected Resource(s)

data "google_project"

Terraform Configuration Files

data "google_project" "sandbox_project" {
  project_id = "sandbox"
}

Debug Output

https://gist.github.com/chimbuchinnadurairightmove/d358e85995dd5083de501cc70ef88d39

Panic Output

Expected Behavior

Terraform should have successfully accessed the cloudbilling.googleapis.com when running the terraform code in GKE pod/GCE VM and returned the billing output

{
    "name": "projects/sandbox/billingInfo",
    "projectId": "sandbox",
    "billingAccountName": "billingAccounts/00000000",
    "billingEnabled": true
}

Actual Behavior

Terraform is unable to access the cloudbilling.googleapis.com and google api returned "Your client does not have permission to get URL" . The same code works when it runs in cloud build / local machine. We have VPC Service control configured and google private access enabled on all the subnets.

Steps to Reproduce

1. Run terraform plan inside GKE pod/ GCE VM

Important Factoids

We are using a dedicated service account and the serviceaccount is having the required permissions. We have also added the serviceaccount to the billing account with billing account viewer permission but still seeing the same behaviour. The same serviceaccount working as expected in cloud build/local.

References

• 0000

[ghost]

We are able to resolve this issue after configuring the cloud DNS response policy for cloudbilling.googleapis.com, The previous private google access setup used the restricted.googleapis.com which is not supporting cloudbilling.googleapis.com. We had to remove the private zone set up for the google private access and configure response policies for cloudbilling.googleapis.com and *.googleapis.com

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.