Time out and crash when retrieving a large number of notifications

[BBonifield]

Sometimes I forget to startup zencoder fetcher before starting a series of jobs. If I try to start it up after the fact to get all of the notifications, zencoder fetcher will consistently crash.

In this particular crash, I was trying to get notifications from 6 jobs that I just submitted with a total of 33 outputs.

bundle exec zencoder_fetcher --api-version v2 -u http://localhost:3000/outputs/update -n 5 -l -m 30 api_key Checking Zencoder for Notifications since Feb 13, 2012 @ 17:51:55 UTC Timeout::Error Timeout::Error /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/protocol.rb:140:in rescue in rbuf_fill' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/protocol.rb:134:inrbuf_fill' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/protocol.rb:116:in readuntil' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/protocol.rb:126:inreadline' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/http.rb:2219:in read_status_line' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/http.rb:2208:inread_new' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/http.rb:1191:in transport_request' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/http.rb:1177:inrequest' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/http.rb:1170:in block in request' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/http.rb:627:instart' /homedir/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/http.rb:1168:in request' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/httparty-0.8.1/lib/httparty/request.rb:73:inperform' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/httparty-0.8.1/lib/httparty.rb:391:in perform_request' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/httparty-0.8.1/lib/httparty.rb:343:inget' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/httparty-0.8.1/lib/httparty.rb:423:in get' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/zencoder-fetcher-0.2.7/lib/zencoder_fetcher.rb:31:inrequest' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/zencoder-fetcher-0.2.7/bin/zencoder_fetcher:66:in block in <top (required)>' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/zencoder-fetcher-0.2.7/bin/zencoder_fetcher:63:inloop' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/gems/zencoder-fetcher-0.2.7/bin/zencoder_fetcher:63:in <top (required)>' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/bin/zencoder_fetcher:19:inload' /homedir/projects/work/sounds-true-media-converter/vendor/bundle/bin/zencoder_fetcher:19:in `

'

[chriswarren]

I'll see if there's anything we can do to make this work better, but for now your best bet is to pass the --count option in your request and limit the number of notifications you receive. It's just timing out because there's a lot of data to gather for those jobs.