Access token refresh failed: invalid_grant: Token has been expired or revoked.

[meet1919]

This is the error I am getting pydrive2.auth.RefreshError: Access token refresh failed: invalid_grant: Token has been expired or revoked. Upon looking at a similar issue @shcheklein mentioned that the error is not in pydrive2 but in pygsheets. I cant find similar issue in the pygsheets.

Below is my code:

gauth.LoadCredentialsFile("mycreds.txt")
if gauth.credentials is None:
    # Authenticate if they're not there
    gauth.LocalWebserverAuth()
elif gauth.access_token_expired:
    # Refresh them if expired
    gauth.Refresh()
else:
    # Initialize the saved creds
    gauth.Authorize()
# Save the current credentials to a file
gauth.SaveCredentialsFile("mycreds.txt")

The seems to work and refresh the token earlier but now it doesnt. Upon looking to the auth.py in pydrive2 I found this:

if self.http is None:
    self.http = self._build_http()
try:
    self.credentials.refresh(self.http)
except AccessTokenRefreshError as error:
    raise RefreshError("Access token refresh failed: %s" % error)

The problem seems to arising at self.credentials.refresh(self.http). I dont know what is self.http but the problem might be coming from self._build_http(). What should I do?

Below is the settings.yaml:

client_config_backend: settings
client_config:
  client_id: id
  client_secret: secret

save_credentials: True
save_credentials_backend: file
save_credentials_file: credentials.json

get_refresh_token: True

oauth_scope:
  - https://www.googleapis.com/auth/drive.file
  - https://www.googleapis.com/auth/drive.install
  - https://www.googleapis.com/auth/drive.metadata

[meet1919]

@shcheklein hey, is there any solution for that? I have a website in production and the working of some features of that website depends on the Drive API. This error pydrive2.auth.RefreshError: Access token refresh failed: invalid_grant: Token has been expired or revoked doesn't seem to go and is also causing setbacks in the some of the processes in the website as I have to manually upload new mycreds.txt file everytime this error comes. Because the gauth.LocalWebServerAuth() doesnt work on the production website, I have to generate it locally and upload the mycreds.txt file every twice in a week.

[shcheklein]

@meet1919 can it be because of this https://developers.google.com/identity/protocols/oauth2#expiration ?

A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of "Testing" is issued a refresh token expiring in 7 days.

[meet1919]

Yes, I think it might be. Both the condition defines my app: External & Testing. Is there a roundabout for this?

[shcheklein]

@meet1919 do you have to use LocalWebServerAuth? do you actually collect people's credentials?

May be you should try service account auth flow? It should be easier (and I'm not sure, but may be it doesn't expire).

https://github.com/iterative/PyDrive2/issues/21

[meet1919]

@shcheklein No LocalWebServerAuth won't work for me as I am using the app for the production. What is from oauth2client.service_account import ServiceAccountCredentials. What is the use of service account used for in the google api?

Anyways I used the code below but it doesn't work for me. I am getting an empty list when printing file_list. Can you point out what is wrong here?

from pydrive2.auth import GoogleAuth
from pydrive2.drive import GoogleDrive
from oauth2client.service_account import ServiceAccountCredentials

scope = ["https://www.googleapis.com/auth/drive"]
gauth = GoogleAuth()
gauth.auth_method = 'service'
gauth.credentials = ServiceAccountCredentials.from_json_keyfile_name('client_secrets.json', scope)
drive = GoogleDrive(gauth)

about = drive.GetAbout()

print('Current user name:{}'.format(about['name']))
print('Root folder ID:{}'.format(about['rootFolderId']))
print('Total quota (bytes):{}'.format(about['quotaBytesTotal']))
print('Used quota (bytes):{}'.format(about['quotaBytesUsed']))

file_list = drive.ListFile({'q': "trashed=false"}).GetList()
print(file_list)

[shcheklein]

@meet1919 first please create a service account credentials as described here - https://dvc.org/doc/user-guide/setup-google-drive-remote#using-service-accounts. It's a separate JSON file. After that you should be able to use this code actually to manage this:

https://github.com/iterative/PyDrive2/blob/main/pydrive2/fs/spec.py#L132

[meet1919]

@shcheklein I have created service account. Do I need to rename the JSON file to client_secrets.json? Also I don't fully understand, how to use the reference you sent me. def _service_auth().

I wrote something like this:

gauth = GoogleAuth()
gauth.credentials = spec._service_auth()
drive = GoogleDrive(gauth)

For the above method involving

from oauth2client.service_account import ServiceAccountCredentials

I need to share the folders to my service account id to make it work. But in that when I use:

drive.ListFile({'q': "'root' in parents and trashed=false"}).GetList()   

I get an empty file_list

[shcheklein]

I meant that you need something like this:

    settings = {
         "get_refresh_token": True,
         "oauth_scope": [
              "https://www.googleapis.com/auth/drive",
              "https://www.googleapis.com/auth/drive.appdata",
        ],
        "client_config_backend": "service",
        "service_config": {
            "client_json": client_json,

            # or:

            "client_json_file_path": client_json_file_path,
        },
    }

    auth = GoogleAuth(settings=settings)

Do I need to rename the JSON file to client_secrets.json

I think a better name is "credentials.json". But it can be anything in this case, you need to specify client_json_file_path or if you load it into a dict, you can pass it via client_json.

I need to share the folders to my service account id to make it work. But in that when I use:

Yes, it needs access. Ot you can enable https://developers.google.com/admin-sdk/directory/v1/guides/delegation and access on behalf of a user, by also providing "service_config.client_user_email".

[meet1919]

@shcheklein thank you for these another method but still it wont work for me as to enable https://developers.google.com/admin-sdk/directory/v1/guides/delegation I need a google administrative account to give access of all the files and folders in the drive to the GCP service account. I think to automatically refresh the token I need to develop an automation testing script (selenium) for the GCP testing account.

[shcheklein]

Delegation is optional. Does it work w/o delegating? You can give access to your service account email to the files you need it to have access to.

I'm not sure what else we can do here. Automatically refreshing it might work, but sounds complicated. You can ask Google btw to make your app production.

[meet1919]

from pydrive2.auth import GoogleAuth
from pydrive2.drive import GoogleDrive
from pydrive2.fs import spec
from oauth2client.service_account import ServiceAccountCredentials

scope = ["https://www.googleapis.com/auth/drive"]
gauth = GoogleAuth()
gauth.auth_method = 'service'
gauth.credentials = ServiceAccountCredentials.from_json_keyfile_name('client_secrets.json', scope)

drive = GoogleDrive(gauth)

folders = drive.ListFile({'q': "'root' in parents and trashed=false"}).GetList()
print(len(folders))

Using this above script, I am getting 0 length of folders and many of my existing scripts use the above query to filter the data on the drive. Automatically refreshing is very complicated. It's not a 100% proof solution. Changing the app to production might be the best solution 💯

[shcheklein]

Each service account has it's own email. It's something long and weird. Have you tried to give access to this "email" to the Google Drive?

[shcheklein]

I mean just go and via UI share folders, files, or the whole drive.

[meet1919]

I have already shared all the folders in the drive to the service account. I think it might work. I am skeptical here because with the normal method OAuth2 credentials I am getting few more files than the service account credentials.

[shcheklein]

I am getting few more files than the service account credentials.

that's weird, could double check that service account has access to them?

[meet1919]

Yes, but I think it can work. With service account I am able to do all the things I was doing with the oauth2 client. Thank you for your suggestion. I just need to change some part of the code and it is set.