Rishikant181 / Rettiwt-API

A CLI tool and an API for fetching data from Twitter for free!
https://rishikant181.github.io/Rettiwt-API/
MIT License
491 stars 46 forks source link

Return the Retry-After or similar headers when rate limited #516

Open EnzoRobaina opened 7 months ago

EnzoRobaina commented 7 months ago
async fetchTweets(user: MyUserInterface) {
    const fetchUserTimeline = async () => {
      const timeline = (await this.client.user.timeline(user.id))
        .list as unknown as RettiwtTweet[]

      return await this.transformTweets(timeline)
    }

    return await backOff(fetchUserTimeline, {
      numOfAttempts: this.maxAttempts,
      startingDelay: this.delay,
      delayFirstAttempt: true,
      retry: (error: any, attemptNumber: number) => {
        if (error.status === 429) {
          console.log(
            `Attempt ${attemptNumber}: Retrying after rate limit error for timeline of ${user.username}`
          )
          return true
        }
        return false
      },
      timeMultiple: this.backoffFactor,
    })
  }

Would it be possible to return how much time must be waited before trying again? This would be very useful when using in conjunction with some backoff/retry implementations

My suggestion for the error object, given that Twitter does provide this info:

{ message: 'TOO_MANY_REQUESTS', status: 429, stack: '...', retryAfter: 2000, }

Rishikant181 commented 7 months ago

I think it's possible. Working on it now.

Rishikant181 commented 7 months ago

@EnzoRobaina What you can do in the time being is, while creating an instance of Rettiwt, you can pass a custom error handler class which implements the IErrorHandler interface. Then handle method can be implemented such that on Error 429, it returns the retry after value.

The implemented error handler can be passed as follows:

// 'CustomErrorHandler' is the class written by you that implements 'IErrorHandler' interface
const handler = new CustomErrorHandler();

const rettiwt = new Rettiwt({ apiKey: API_KEY, errorHandler: handler });
EnzoRobaina commented 7 months ago

Thats great! Thanks for the quick reply!