Unable to use this provider, always 'Request had insufficient authentication scopes'

[lhriley]

Hello,

I've been stuck on this for a while now, and I'm really not clear on what I could be doing wrong. I've tried setting up my credentials based on the documentation here: https://github.com/DeviaVir/terraform-provider-gsuite/blob/v0.1.58/website/docs/index.html.markdown#best-practices

I've tried using a service account (as recommended) as well as my own account using the credentials from gcloud auth login. My account is a Super Admin in both Google Workspace as well as GCP, and I have successfully used the gcloud CLI, Python and Terraform to manage resources in GCP. The only time I run into an issue is using this gsuite provider.

Additionally, I've used many different combinations of the oauth_scopes as documented. Further, I've tried completely unsetting the oauth_scopes as suggested when using a personal admin account. No combination seems to make any difference.

No matter what I do or do not pass to the provider directly or through environment variables I always get the error:

[ERROR] Error creating group: googleapi: Error 403: Request had insufficient authentication scopes.

In the debug logs the response always looks like this:

021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: 2021/03/22 15:56:13 [DEBUG] Google API Response Details:
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: ---[ RESPONSE ]--------------------------------------
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: HTTP/2.0 403 Forbidden
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Cache-Control: private
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Content-Type: application/json; charset=UTF-8
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Date: Mon, 22 Mar 2021 22:56:13 GMT
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Server: ESF
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Vary: Origin
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Vary: X-Origin
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Vary: Referer
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: Www-Authenticate: Bearer realm="https://accounts.google.com/", error="insufficient_scope", scope="https://apps-apis.google.com/a/feeds/groups/ https://www.googleapis.com/auth/admin.directory.group https://www.googleapis.com/auth/directory.group"
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: X-Content-Type-Options: nosniff
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: X-Frame-Options: SAMEORIGIN
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: X-Xss-Protection: 0
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: 
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: {
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:   "error": {
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:     "code": 403,
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:     "message": "Request had insufficient authentication scopes.",
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:     "errors": [
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:       {
on",
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:         "domain": "global",
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:         "reason": "insufficientPermissions"
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:       }
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:     ],
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:     "status": "PERMISSION_DENIED"
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58:   }
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: }
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: 
2021-03-22T15:56:13.877-0700 [DEBUG] plugin.terraform-provider-gsuite_v0.1.58: -----------------------------------------------------

[codedbypm]

I believe what is crucial is the fact that you can't directly access the Directory API from GCP. Only a Google Workspace user with enough rights can do that. And that's why the GCP account you use to access the API (wether a user or a service account) must impersonate that Google Workspace user. You achieve that by setting the correct impersonated user email when you config the plugin. If I were you, I'd double-check that.

[lhriley]

@codedbypm thanks for the response. I feel like I should have rights to access the API, which is why I'm confused. I've gone over the related documentation here and on Google's side several times, and verified that everything is setup correctly (as best I can tell, anyways).

Do you know of a simple example for validating the correct access rights? For example, a gcloud command?

[codedbypm]

Unfortunately I don't think there is a gcloud command for that. But can't confirm that. I have access to admin.google.com (the gsuite admin portal) and there I had to create a special user who has admin rights on the Users and Groups API. You don't have to create such a user by the way. What is important is that you have at least one user with those admin rights. And that's the user you impersonate when using this plugin.

[lhriley]

@codedbypm thanks for the additional information, and sorry for the delayed response.

Just to confirm this bit:

What is important is that you have at least one user with those admin rights. And that's the user you impersonate when using this plugin.

This should definitely be the case as my personal account has these rights as verified in the roles page of my Google Workspace user. This is the same account that I setup via gcloud which grants me admin rights in Google Workspace and GCP.

Are you not using a GCP service account at all for this task; just the Google Workspace account?

---

Today I finally found some time to try this again. I removed all of the services accounts, domain wide delegation integration, etc that I created as part of the initial setup and started from scratch. Unfortunately, I ended up in the exact same situation with the same error.

One new piece that I don't recall running into before is now when I set the impersonated email address, I get a different error. This time it looks like it is trying to contact the GCP metadata server and fails. I'm running everything from my laptop locally, and I'm not sure why this happens when setting the impersonated email address. Perhaps it assumes this option is only used when running from within GCP? I haven't yet found any options that are associated with this behaviour.

Error: failed to load config: failed to get service account from metadata server: Get "http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/email": dial tcp 169.254.169.254:80: i/o timeout

If it isn't too much trouble and you're using the GCP service account integration path, could you possibly share the Oauth scopes you're assigning in Google Workspace Domain-wide Delegation for you API client? I'm using the following as described as minimum requirements in the documentation:

https://www.googleapis.com/auth/admin.directory.user
https://www.googleapis.com/auth/admin.directory.group

I would assume that if the impersonation email worked correctly, I would be using the service account linked to the API client ID that I created following the Google documentation which would be similar to <some-id>@<some-project>.gserviceaccount.com. Just making sure I'm not completely off base here.

[DeviaVir]

That is indeed new per https://github.com/DeviaVir/terraform-provider-gsuite/pull/171

[codedbypm]

Are you not using a GCP service account at all for this task; just the Google Workspace account?

I use a service account from my GCP project, which has Domain Wide Delegation enabled. These are the scopes I set in the Admin console:

This is instead a special GWorkspace user I created that has access to the Directory API. Its email address is the one I fill in for the impersonated_email parameter of this terraform plugin. My use case is to be able to create/delete GWorkspace users from terraform. And this setup works fine.

[lhriley]

@codedbypm Thanks for the details, and sorry for the delay. The notification got lost in my email =\

Hmm. This is very odd. I feel like what you're showing me is exactly what I have done aside from the dedicated account for impersonation. In any case it is helpful to see a working example. I might try to exactly replicate what you've demonstrated to see if that solves the problem, then work backwards to see what I'm missing.

@DeviaVir Thanks for clarifying. The new error would probably be more helpful in tracking down where the process is failing at the very least.

Thanks again for the assistance!

[jazzlyn]

@lhriley hi, not sure what exactly is the problem but I wrote an example of how to authenticate with gworkspace using this provider. https://github.com/strg-at/terraform-gsuite-user-group-management/tree/main/examples/example-gsuite-authentication for the gsuite_domain you will need the additional scope: https://www.googleapis.com/auth/admin.directory.domain

The user account needs admin permissions (e.g. Super admin or at least Groups- and User management admin, see https://support.google.com/a/answer/2405986?hl=en)

maybe this helps with debugging.

[lhriley]

@jazzlyn Thanks for the example to work from. I'm also unclear what the root cause of this issue is.

I'll test out your example code once I get the opportunity to, but I'm actually starting a vacation tomorrow so it might be a while before I circle back to this.

Thanks again.

[dlethin]

I also experienced this issue and struggled for several hours wrestling with it. In the end I found a solution I can share in case it helps others:

I specifically wanted to use a user account which had sufficient admin privs in my workspace rather than use a service account. For my localized development of this project, it seemed like a good idea to use short-lived credentials, and that is echoed in the following links:

• https://medium.com/google-cloud/local-remote-authentication-with-google-cloud-platform-afe3aa017b95
• https://jryancanty.medium.com/stop-downloading-google-cloud-service-account-keys-1811d44a97d9

Cutting to the punchline, what worked for me was making sure to include the --client-id-file and -scope command line arguments when running gcloud auth application-default login, as in this example:

✗ gcloud auth application-default login --no-launch-browser --scopes="https://www.googleapis.com/auth/admin.directory.usersreadonly" --client-id-file ~/path/to/client_secret.json

_In my case above I just started experimenting with this provider using the gsuite_user data source, so setting my scope to a readonly variant was fine_

Without specifying the --client-id-file parameter, the browser page you get directed to will reference the project "Google Auth Library". When you include that param and point it to a json file containing your specific GCP project's OAuth 2.0 Client ID and secret, you will get redirected to an auth page correctly showing your project's name and the auth scope you're requesting.

This link was helpful for setting up OAuth2 and finding access to the project's client id and secret.

------ edit 1:

I meant to point out that after performing the above command, my project worked with a very simple provider config -- No need for any arguments:

terraform {
  required_providers {
    gsuite = {
      source = "DeviaVir/gsuite"
      version = "0.1.62"
    }
  }
}

provider "gsuite" {}

variable "user" {}

data "gsuite_user" "user" {
  primary_email = var.user
}

---

Hope that provides enough breadcrumbs to help others out.

Cheers