sferik / twitter-ruby

A Ruby interface to the Twitter API.
http://www.rubydoc.info/gems/twitter
MIT License
4.58k stars 1.31k forks source link

How do I use twitter cursor with this gem? #963

Closed Kayracz closed 1 year ago

Kayracz commented 4 years ago

Hey Problem: Need to get 1000+ followers names.,

I've implemented getting followers list using your gem, but looks like I can get only 300 followers in one go. I understand that, Twitter has 15 request per 15 mins and each request will give me 20 followers at a time. I've few users who have more than 1000 followers, how do I get the names of the followers?

Currently, I'm using following code,

twitter_response = TwitterAPI.new

followers = twitter_response.client(current_user).followers

followers.each do |follower|

     f = Follower.create(name: follower.name, user_id: follower.id)

end

Now after making 20 request , I encounter Twitter::Error::TooManyRequests: Rate limit exceeded error. I read it online, that I can use followers.attrs[:next_cursor] to get data offline. Can you please share an example to do that using this gem?

Or if there is a better way to solve the issue, I'm all ears.

AgoraSecurity commented 3 years ago

Hello,

If you still have the doubt, I feel the following code can solve your issue:

# Configure Twitter::REST::Client
@client = Twitter::REST::Client.new do |config|
    config.consumer_key        = twitter_tokens.delete(:key)
    config.consumer_secret     = twitter_tokens.delete(:secret)
    config.access_token        = twitter_tokens.delete(:token)
    config.access_token_secret = twitter_tokens.delete(:token_secret)
end

# We'll store the followers IDs
all_followers_ids = []

cursor = nil

# Use loop. Better methods available.
loop do
    options = { count: 50 }
    unless cursor.nil?
        options[:no_default_cursor] = true
        options[:cursor] = cursor
    end
    begin
        followers = @client.followers(options)
    rescue Twitter::Error::TooManyRequests => error
        # NOTE: Your process could go to sleep for up to 15 minutes but if you
        # retry any sooner, it will almost certainly fail with the same exception.
        sleep error.rate_limit.reset_in + 1
        retry
    end
    all_followers_ids << followers.to_h[:users].map {|f| f[:id]}
    cursor = followers.to_h[:next_cursor]
    # Stop if we've reach the end
    break if cursor.nil?
end

all_followers_ids.flatten

Note: It was a quick ruby code. If you need this for a production app, I suggest you refactor it :)