Mechanize::ResponseCodeError: 429 => Net::HTTPTooManyRequests

sparklemotion / mechanize

Mechanize is a ruby library that makes automated web interaction easy.

https://www.rubydoc.info/gems/mechanize/

MIT License

4.39k stars 473 forks source link

Mechanize::ResponseCodeError: 429 => Net::HTTPTooManyRequests #572

Closed wonderer007 closed 3 years ago

wonderer007 commented 3 years ago

I am writing a bot to scrape a website and within x time frame server respond with the below error

Mechanize::ResponseCodeError: 429 => Net::HTTPTooManyRequests for https://www.myscrappingserver.com -- unhandled response

After Y amount of time, I can make a request to the server again. Can I avoid this behaviour by using proxy with Mechanize gem or changing user agent?