Value passed for token in Next.js app is Invalid

[jacklynch00]

I am using the twitter api sdk in a Next.js server-less function and I am currently creating a new twitter Client on each request, which works for the first request but fails every time after until I get a new access token.

Expected behavior

I expect to be able to make multiple requests with the Twitter API without having to re-authenticate and get a new API Access Token.

Actual behavior

Step 1 - Authenticate

• Using Next.JS's next-auth package, I authenticate with the Twitter API and store the access and refresh tokens in a JWT.

Step 2 - Use Next.js server-less function to create Twitter Client and make a request

• I then make a request to the next.js server-less function which get the users JWT, grabs the needed information (tokens, scope, etc.), and makes the request to the Twitter API

export default async function twitterTweet(req: NextApiRequest, res: NextApiResponse) {
    const session = await getSession({ req });
    const token = await getToken({ req, secret: process.env.NEXTAUTH_SECRET });

    if (!session) {
        return res.status(401).end('Not authenticated');
    }

    const tokenInfo = hasTwitterConfig(token) ? token : null;
    if (!tokenInfo) {
        return res.status(400).end('Access token not provided');
    }

    const twitterClient = initTwitterClient(tokenInfo.twitter);

    const tweet = await twitterClient.tweets
        .createTweet({
            text: 'This is a test tweet',
        });

    return res.status(200).json(tweet);
}

Step 3 - Encounter error after making one request

• I am able to successfully complete the above request one time with no issues and I am able to create a tweet. However, if I try and do it again, there is an error thrown by the Twitter SDK indicating an issue with the token (even though it just worked)

error - TwitterResponseError
    at request (/Users/jack/code/threadifier/node_modules/twitter-api-sdk/dist/request.js:67:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async rest (/Users/jack/code/threadifier/node_modules/twitter-api-sdk/dist/request.js:100:22)
    at async OAuth2User.refreshAccessToken (/Users/jack/code/threadifier/node_modules/twitter-api-sdk/dist/OAuth2User.js:67:22)
    at async OAuth2User.getAuthHeader (/Users/jack/code/threadifier/node_modules/twitter-api-sdk/dist/OAuth2User.js:208:13)
    at async request (/Users/jack/code/threadifier/node_modules/twitter-api-sdk/dist/request.js:48:11)
    at async rest (/Users/jack/code/threadifier/node_modules/twitter-api-sdk/dist/request.js:100:22)
    at async twitterTweet (webpack-internal:///(api)/./pages/api/twitter/tweet/all.ts:32:19)
    at async Object.apiResolver (/Users/jack/code/threadifier/node_modules/next/dist/server/api-utils/node.js:363:9) {
  status: 400,
  statusText: 'Bad Request',
  headers: {
    'cache-control': 'no-cache, no-store, max-age=0',
    connection: 'close',
    'content-disposition': 'attachment; filename=json.json',
    'content-encoding': 'gzip',
    'content-length': '103',
    'content-type': 'application/json;charset=UTF-8',
    date: 'Sun, 04 Dec 2022 20:52:24 GMT',
    perf: '7626143928',
    server: 'tsa_b',
    'set-cookie': 'guest_id_marketing=v1%3A167018714457757711; Max-Age=63072000; Expires=Tue, 03 Dec 2024 20:52:24 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None, guest_id_ads=v1%3A167018714457757711; Max-Age=63072000; Expires=Tue, 03 Dec 2024 20:52:24 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None, personalization_id="v1_3Yqtlpgf4Rlqo2sBprv9Cw=="; Max-Age=63072000; Expires=Tue, 03 Dec 2024 20:52:24 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None, guest_id=v1%3A167018714457757711; Max-Age=63072000; Expires=Tue, 03 Dec 2024 20:52:24 GMT; Path=/; Domain=.twitter.com; Secure; SameSite=None',
    'strict-transport-security': 'max-age=631138519',
    'x-connection-hash': 'ae9d1813834f482cc556e67b27eb51207eb17eeb2f9888e0c8f3cbbb8a19ce80',
    'x-content-type-options': 'nosniff',
    'x-frame-options': 'SAMEORIGIN',
    'x-response-time': '17',
    'x-transaction-id': '5d99f8a156f432bc',
    'x-xss-protection': '0'
  },
  error: {
    error: 'invalid_request',
    error_description: 'Value passed for the token was invalid.'
  },
  page: '/api/twitter/tweet/all'
}

Steps to reproduce the behavior

Listed the steps above

[Th3Gavst3r]

I ran into this problem too. It happens because the authClient automatically exchanges both your access_token and refresh_token when you don't construct it with a value for expires_at. See here.

After it exchanges your tokens, the ones you have saved in your app will be invalidated and invalid_request will be thrown the next time you create an authClient.

To avoid this problem, you have to update wherever you're storing your tokens when they change. Unfortunately this project doesn't provide a way to monitor the tokens, so you'll need a wrapper to keep track of changes.

[jacklynch00]

So are you suggesting storing the access_token and refresh_token in the next-auth token?

[Th3Gavst3r]

I don't know the specifics about Next.js, but I decided to store mine in an encrypted session cookie. You just have to make sure your cross-request state is kept up to date.

[jacklynch00]

So then you're saying that you set the access_token and refresh_token every time you use the authClient to make a request?

[Th3Gavst3r]

I didn't set the token every time because if you do (and don't provide a value for expires_at) you'll be unnecessarily sending token refresh requests on every API call. Once it's constructed the authClient is able to manage its own state fine, so I just wrote a wrapper around my Client which monitors its authClient and updates the user's session data when the token changes.

[jacklynch00]

You have been very helpful so thank you for that @Th3Gavst3r !! Would you happen to have any sort of example code somewher you could point me towards though? I am struggling to figure out the abstraction to a wrapper around the authClient as it related to my specific use case within NextJS (but I think some type of example would be better than nothing).

[Th3Gavst3r]

Essentially, I just created a class that takes an authClient and tokenCallback as constructor parameters. Then I re-exposed the endpoint functions I needed for my app and ran the tokenCallback after every call.

// twitter-api-typescript-sdk does not expose a type definition for this
export interface Token {
  access_token?: string;
  refresh_token?: string;
  expires_at?: number;
}

export default class TwitterService {
  private readonly MAX_RETRIES = 3;
  private token?: Token;
  private client: Client;

  constructor(
    private readonly authClient: OAuth2User,
    private readonly tokenCallback: (token: Token | undefined) => Promise<void>
  ) {
    this.token = authClient.token;
    this.client = new Client(authClient);
  }

  private async checkForUpdatedToken(): Promise<void> {
    if (this.authClient.token !== this.token) {
      this.token = this.authClient.token;
      await this.tokenCallback(this.token);
    }
  }

  public async findUserByUsername(username: string) {
    const usernameResult = await this.client.users.findUserByUsername(
      username,
      {
        'user.fields': ['created_at'],
      },
      {
        max_retries: this.MAX_RETRIES,
      }
    );
    await this.checkForUpdatedToken();
    return usernameResult;
  }
}

const token = getExistingTokenFromDatabase();
const myAuthClient = new auth.OAuth2User({
  ...
  token: token,
});
const twitterService = new TwitterService(myAuthClient, (token) => {
  saveNewTokenToDatabase(token);
});
const myUser = twitterService.findUserByUsername('Th3Gavst3r');

[feresr]

Once it's constructed the authClient is able to manage its own state fine

OMG thank you @Th3Gavst3r you've helped me tremendously. Calling checkForUpdatedToken() AFTER using the client feels counter-intuitive but is the right thing to do.

const usernameResult = await this.client.users.findUserByUsername(username, {...});
await this.checkForUpdatedToken();

I ended up with

export const withClient = async <T>(
  session: IronSession<SessionData>,
  callback: (client: Client) => T
): Promise<T> => {
  const user: OAuth2User = createUser(session.token);
  // The client refreshes the token internally on its own.
  const result = callback(new Client(user));
  // Check if the token changed, update the session if so.
  if (session.token != user.token) {
    session.token = user.token!;
    await session.save();
  }
  return result;
};

// usage
const user_data = await withClient(session, async (client: Client) => {
    // use client as needed
    return data;
});