Support for Twitter API v2 with OAuth 2.0

[ash0080]

Instead of jumping back to the /callback, it goes directly to /token, so it looks like an internal bug, I've set up google, facebook, github, and they all work fine.

I tried examples/response-profile, and it's the same issue

I looked at the doc for twitter oauth 1.0a and found that twitter is different from other oauths in that its callback is a query string and what in the dashboard is set as a whitelist, so I'm guessing there's a miss-concating of the query string?

2022-01-16T17:30:49.860Z *37920 @ElvisMini.local -  info: #req-6 ←GET:/oauth/twitter request from ip 127.0.0.1 (fastify-request-logger)
2022-01-16T17:30:50.813Z *37920 @ElvisMini.local -  info: #req-6 →GET:/oauth/twitter response with a 302-status (fastify-request-logger)
2022-01-16T17:30:50.821Z *37920 @ElvisMini.local -  info: #req-7 ←GET:/token request from ip 127.0.0.1 (fastify-request-logger)
{
  provider: 'twitter',
  request: {
    error: {
      errors: [ { code: 32, message: 'Could not authenticate you.' } ]
    }
  },
  response: {
    error: {
      errors: [ { code: 32, message: 'Could not authenticate you.' } ]
    }
  }
}
2022-01-16T17:30:50.822Z *37920 @ElvisMini.local -  info: #req-7 →GET:/token response with a 200-status (fastify-request-logger)

[simov]

Can you share your Grant configuration as well (without sensitive data in it)?

[ash0080]

Can you share your Grant configuration as well (without sensitive data in it)?

twitter : {
    key   : process.env.TWITTER_CLIENT_ID,
    secret: process.env.TWITTER_CLIENT_SECRET,
    //scope        : ['email', 'public_profile', 'offline.access'],
    //response: ['tokens', 'profile'],
    //custom_params: {access_type: 'offline'},
    callback: '/token',
  },

[simov]

I mean the entire config.

[ash0080]

module.exports = {
  defaults: {
    origin: 'http://localhost:3009', prefix: '/oauth', transport: 'session', state: true, nonce: true,
  },
  github  : {
    key          : process.env.GITHUB_CLIENT_ID,
    secret       : process.env.GITHUB_CLIENT_SECRET,
    scope        : ['read:user', 'user:email', 'offline_access'],
    response     : ['tokens', 'raw', 'jwt', 'profile'],
    custom_params: {'access_type': 'offline'},
    callback     : '/token',
  },
  google  : {
    key          : process.env.GOOGLE_CLIENT_ID,
    secret       : process.env.GOOGLE_CLIENT_SECRET, 
    access_url   : 'https://oauth2.googleapis.com/token',
    scope        : ['email', 'profile'],
    custom_params: {access_type: 'offline'},
    response     : ['tokens', 'profile', 'raw'],
    callback     : '/token',
  },
  facebook: {
    key          : process.env.FACEBOOK_CLIENT_ID,
    secret       : process.env.FACEBOOK_CLIENT_SECRET,
    scope        : ['email', 'public_profile'],
    custom_params: {access_type: 'offline'},
    response     : ['tokens', 'profile'],
    callback     : '/token',
  },
  twitter : {
    key   : process.env.TWITTER_CLIENT_ID,
    secret: process.env.TWITTER_CLIENT_SECRET,
    //scope        : ['email', 'public_profile', 'offline.access'],
    //response: ['tokens', 'profile'],
    //scope        : [],
    //custom_params: {access_type: 'offline'},
    //response     : [],
    callback: '/token',
  }
}

twitter doesn't support localhost, but its doc says it supports custom local domain, so I did that by modifying the hosts file.

By the way, for google, if I use the original access_url, there is a certain chance that the profile will not be fetched. for facebook, now there is a high probability of not getting a profile (80% for me), but I think this may be a problem from facebook, but if there is a retry mechanism, it may be improved.

[simov]

You have to update your origin as well or hardcode the redirect_uri:

{
  "defaults": {
    "origin": "http://sealight:3009"
  }
}

or

{
  "twitter": {
    "redirect_uri": "http://sealight:3009/oauth/twitter/callback"
  }
}

see the documentation about it.

For Google you need to update the URL as you found out in order to get the OpenID profile. I have a PR here for it https://github.com/simov/grant/pull/266

Facebook is not an OpenID Connect provider, meaning scopes like email are not available, or at least they were not last time I checked. So probably it is the public_profile instead. In case you took that from their docs then I don't see a reason why this won't work.

Note that access_type: offline may not be available for GitHub and Facebook as it is with Google. Not all of these parameters are part of a specification and even if they are some providers may still decide whether to support them or not.

[ash0080]

You have to update your origin as well or hardcode the redirect_uri:

{
  "defaults": {
    "origin": "http://sealight:3009"
  }
}

or

{
  "twitter": {
    "redirect_uri": "http://sealight:3009/oauth/twitter/callback"
  }
}

Thanks @simov, But I actually tried both of these yesterday, just a few minutes after submitting the issue, and neither worked. Even if I override redirect_uri, it still jumps straight back to /token, which is why I think it's a string concating error.

[ash0080]

@simov update: I ended up using ngrok, and now it's a full online environment, but the symptoms are still the same.

[simov]

That was the issue with your config, your redirect URI for Twitter was incorrect. I can't think of anything else right now other than some other type of misconfiguration on your end. There is no concatenation error, logging in with Twitter works on my end as well.

[ash0080]

@simov I tried debugging and found this,

This is twitter's doc

curl --request POST \
  --url 'https://api.twitter.com/oauth/request_token?oauth_callback=$HTTP_ENCODED_CALLBACK_URL' \
  --header 'Authorization: OAuth oauth_consumer_key="$oauth_consumer_key", oauth_nonce="$oauth_nonce", oauth_signature="oauth_signature", oauth_signature_method="HMAC-SHA1", oauth_timestamp="$timestamp", oauth_version="1.0"'

Code copied to clipboard

This is request-oauth actual Sent, see the different? in Twitter's doc, oauth_callback is a query string, But in request-oauth, it's in the headers, Could this be the problem?

[simov]

No, there is no bug in the underlying library or in Grant.

[ash0080]

No, there is no bug in the underlying library or in Grant.

@simov Never mind, I got it. Twitter now has an OAuth 2 key secret pair and a consumer key pair, I used the wrong pair, there is some confusion between the naming.

[simov]

That's a good point actually .. I have not checked the Twitter API in awhile. Historically they had only OAuth 1.0a supported. As for Grant there is no difference between OAuth 1.0a or OAuth 2.0 that's why you can set your credentials as simply key and secret. But you can use the longer form if you want to, check out the Client App section in this table https://github.com/simov/grant#configuration-description

[ash0080]

@simov Very sorry to bother you again, I met a new problem, I upgraded twitter to elevated level a little while ago， Because the 1.1a Essential level no longer has access to the user profile， But, After I upgraded, I found that I still couldn't get the user profile properly

  key     : process.env.TWITTER_CLIENT_ID,
  secret  : process.env.TWITTER_CLIENT_SECRET,
  response: ['tokens', 'raw'], 
  callback: '/token',

under this config, nothing special, I got id and screen_name under raw

  key     : process.env.TWITTER_CLIENT_ID,
  secret  : process.env.TWITTER_CLIENT_SECRET,
  response: ['tokens', 'raw', 'profile'], 
  callback: '/token',

but after add 'profile', I got this response

{
  provider: 'twitter',
  request: {
    oauth_token: 'masked',
    oauth_token_secret: 'masked',
    oauth_callback_confirmed: 'true'
  },
  response: 'https://api.twitter.com/oauth/authenticate?oauth_token=masked'
}

[simov]

Can you send me the link to that page in Twitter's developer documentation, so that I can try to create an app for myself and see how that goes. I'm not sure what those app levels are tbh, there might be something new in their API.

[ash0080]

Can you send me the link to that page in Twitter's developer documentation, so that I can try to create an app for myself and see how that goes. I'm not sure what those app levels are tbh, there might be something new in their API.

https://developer.twitter.com/en/docs/twitter-api/v1/accounts-and-users/follow-search-get-users/api-reference/get-users-show

[ash0080]

maybe relate this #253

[simov]

I pushed the twitter2 provider that is supposed to be used for Twitter with OAuth 2.0 here https://github.com/simov/grant/commit/101c95ebb105db2839336606912d35fe97ffd569

One thing to note is that these options are required as well:

{
  "twitter2": {
    "state": true,
    "pkce": true,
    "scope": [
      "users.read",
      "tweet.read"
    ]
  }
}

You need at least one scope, but these two are required in order to get the /2/users/me endpoint.

You can reference it as github:simov/grant in your package.json for now as I will have to add some tests for that change that I did.

[ash0080]

I pushed the twitter2 provider that is supposed to be used for Twitter with OAuth 2.0 here 101c95e

One thing to note is that these options are required as well:

{
  "twitter2": {
    "state": true,
    "pkce": true,
    "scope": [
      "users.read",
      "tweet.read"
    ]
  }
}

You need at least one scope, but these two are required in order to get the /2/users/me endpoint.

You can reference it as github:simov/grant in your package.json for now as I will have to add some tests for that change that I did.

Many thanks @simov，It' works I released a project hasura-oauth-service day before yesterday, I will update the dependencies as soon as possible.

There is one area that could be improved profile_url: "https://api.twitter.com/2/users/me?user.fields=id,name,username,profile_image_url,description"

[simov]

Published in v5.4.20