Support refresh_token

[mejiaej]

Hi, I haven't found any documentation related to the expiration of tokens in a google drive upload scenario. I want to know how often a user is gonna need to consent. Google OAuth gives an acces_token which is short lived and a refresh_token with an expiration of 6 months on production.

I haven't found any reference to refresh tokens inside companion, Does companion use refresh_tokens? If not how does companion handle the expiration of short-lived access_token?

[othierry-asi]

I have the same question. Is there a way to have a long authentication between the companion provider and the cloud provider ? What I want to do is ask permissions to the user once, then store the refresh_token and use it further so the user does not need to authenticate every time it wants to upload a file from Dropbox or any storage provider.

[dovydaskukalis]

Similar question. Is there a way to use refresh_token?

[Murderlon]

@mifi are you able to answer this?

[mifi]

Hi, I haven't found any documentation related to the expiration of tokens in a google drive upload scenario. I want to know how often a user is gonna need to consent. Google OAuth gives an acces_token which is short lived and a refresh_token with an expiration of 6 months on production.

Hi. As far as I know (from my testing), the user does not receive a new consent dialog every time the access token is expired, but they just need to reauthenticate. I believe this is because they have already allowed the app to access their account so they just need to choose their google account again (and possibly enter password again).

You are right that the access token lasts a short time (e.g. less than an hour or so) and this is by design / best practice. And companion does not support refresh tokens, and so the user will have to reauthenticate after the short lived access token has expired.

So I consider this a feature request.

Implementation

Refreshing the token

If we were using Google's official Node.js SDK it would automatically handle refreshing the token:

Access tokens expire. This library will automatically use a refresh token to obtain a new access token if it is about to expire. An easy way to make sure you always store the most recent tokens is to use the tokens event:

https://github.com/googleapis/google-api-nodejs-client#handling-refresh-tokens

But because we are using a custom HTTP client (purest), we need to either

• implement this refreshing logic ourselves
• or rewrite the Drive provider to use the official google node sdk. https://github.com/transloadit/uppy/blob/b7559acd7c9142094dd5ab0ca3fe84db48bdf99c/packages/%40uppy/companion/src/server/provider/drive/index.js#L95

Storing the updated refresh token

Upon receiving a refreshed token from google we need to do this:

• replace the access_token and refresh_token inside the JWT with the new ones, repack and encrypt the JWT
• notifiy the client about the new encrypted jwt (maybe a response header on the request's response)
• the client would then need to store the new encrypted jwt so that it will use that in subsequent requests
• must still handle the error if a refresh token has expired or been invalidated

refresh_token/access_token race condition

Need to look into possible race condition:

If there are multiple simultaneous companion requests that all refresh the tokens at the same time, will that "just work"? (can a single refresh_token be refreshed many times and produce many new valid refresh_tokens and access_tokens? or will only the first refresh call succeed, and the rest of the requests will fail, leading to potentially many failed uploads for the user?

I found this:

In multiprocess or distributed apps, sharing the credential requires you to persist it. To ensure multiple processes or servers don't attempt to refresh the credential at the same time (resulting in excessive refresh requests), we advise proactively refreshing the credential in a central place and sharing it across processes/servers.

• https://stackoverflow.com/questions/37084682/is-oauth-thread-safe
• https://developers.google.com/adwords/api/docs/guides/authentication#multiprocess_or_distributed_apps

Possible solutions:

• Proactively refresh the token before it expires and update the client. Old access token should still work for a while, so any other requests should go through. Need to somehow make sure that only one request/server will do this refresh operation. I'm not sure how to best do this.
• Use a request queue on the client or server so that only one http request (per encrypted jwt) can be sent to google at any given time. (e.g. https://github.com/sindresorhus/p-queue)
  • harder to implement on the server (needs to be enforced across multiple horizontally scaled companion instances.)

This also needs to be implemented for all the other companion providers.

[Murderlon]

@mifi thanks for the elaborate research. From the looks of it, this is a lot of effort to implement for questionable gains so I'm inclined to close this for that reason. We need to start prioritizing what we will actually work on at some point, and be real about it when we won't.

cc @arturi

[mifi]

yea, it adds another level of complexity, and refresh tokens differ between all the providers.

[Murderlon]

Alright. Closing this but may revisited and reopened at some point.

[mifi]

reconsidering this - maybe we should do the refreshing of the token in the Uppy client - then we don't need to coordinate multiple refreshing the token between companion servers, but instead we could do it in the client, because there's always just one client.

We would still have to coordinate this between all ongoing requests and RateLimitQueue.

Let's reopen this, because it will effectively prevent any upload sessions longer than about 1 hour.

pseudo code:

// global
let refreshingTokenPromise;

async function downloadFile() {
  try {
    if (refreshingTokenPromise) await refreshingTokenPromise;

    await get('/companion/files/12345')
  } catch (err) {
    if (err.status === 401) {
      refreshingTokenPromise = put('/companion/authtoken/refresh', { refreshToken: getRefreshToken() })
      await refreshingTokenPromise
      refreshingTokenPromise =  undefined
      return downloadFile()
    }
    throw err;
  }  
}