search

chadrem / market_bot

Google Play Android App store scraper
MIT License
361 stars 130 forks source link

market_bot freezes if network goes down #9

Closed guss77 closed 9 years ago

guss77 commented 11 years ago

It appears that market_bot requests freeze and never return if network goes down in the middle of the request.

I've tried sending :request_opts => { :timeout => 5000 } but it doesn't seem to have any effect.

If the network ever comes back, market_bot continues and I eventually get the result.

chadrem commented 11 years ago

I haven't experienced this problem so let me know if you find a solution. :request_opts should let you pass options directly to the typhoeus request object. Have you tested typhoeus (without market_bot) to make sure :timeout works as you expect? Perhaps there is a problem with the version of typhoeus you are using.

guss77 commented 11 years ago

When I use typhoeus directly, and I set the timeout to be the number of seconds (not milliseconds as their documentation suggests) then the request times out if the network is down. Also, new requests fail immediately when the network is down.