search

jefflinwood / Tweeter-Keeper

Ruby and MongoDB integration for the back end of any new Twitter analytics/processing project
32 stars 9 forks source link

What happens if the limit is reached or connection dropped ? #2

Closed ker2x closed 5 years ago

ker2x commented 12 years ago

Friendly greetings ! Thank you very much for this app, it saved me a lot of time (can't say i'm a good coder).

I see that in your TODO:

What's hapenning right now on error or rate limit ? The connection is dropped and there is no auto-recovery ? (that's what i understand by reading the TODO).

Any plan to add thoses 2 points in the next few weeks ? :) Thank you !!!

jefflinwood commented 12 years ago

Hi,

I haven't really been working with this little Ruby script recently, having moved to Twitter's storm framework for pulling content off the Twitter Streaming API and then doing a map/reduce in real-time. Of course, that's a lot more complicated!

Are you running into problems with it?

Jeff Linwood

ker2x commented 12 years ago

Just got one problem :)

/home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/tweetstream-1.1.1/lib/tweetstream/client.rb:367:in block (2 levels) in start': Failed to reconnect after 7 tries. (TweetStream::Recon nectError) from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/twitter-stream-0.1.14/lib/twitter/json_stream.rb:140:incall' from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/twitter-stream-0.1.14/lib/twitter/json_stream.rb:140:in schedule_reconnect' from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/twitter-stream-0.1.14/lib/twitter/json_stream.rb:113:inunbind' from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-0.12.10/lib/eventmachine.rb:1417:in event_callback' from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:inrun_machine' from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in run' from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/tweetstream-1.1.1/lib/tweetstream/client.rb:310:instart' from /home/ker2x/.rvm/gems/ruby-1.9.2-p290/gems/tweetstream-1.1.1/lib/tweetstream/client.rb:111:in `filter'

jefflinwood commented 12 years ago

Yes - so TwitterStream would need to do is handle the error, backing off of reconnect errors along the way, so it doesn't try to flood Twitter with connections.