search

yatish27 / linkedin-scraper

Scrapes the public profile of the linkedin page
MIT License
552 stars 221 forks source link

999 response error while running from racksapce #39

Closed carlotorniai closed 9 years ago

carlotorniai commented 9 years ago

HI I using the scraper form my Rackspece machine I receive a 999 response. Details of the error below: linkedin-scraper http://www.linkedin.com/in/jeffweiner08 /usr/local/rvm/gems/ruby-2.0.0-p247/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:933:in response_read': 999 => for -- http://www.linkedin.com/in/jeffweiner08 (Mechanize::ResponseCodeError) from /usr/local/rvm/gems/ruby-2.0.0-p247/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:262:inblock in fetch' from /usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1413:in block (2 levels) in transport_request' from /usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http/response.rb:162:inreading_body' from /usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1412:in block in transport_request' from /usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1403:incatch' from /usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1403:in transport_request' from /usr/local/rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1376:inrequest' from /usr/local/rvm/gems/ruby-2.0.0-p247/gems/net-http-persistent-2.9/lib/net/http/persistent.rb:986:in request' from /usr/local/rvm/gems/ruby-2.0.0-p247/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:259:infetch' from /usr/local/rvm/gems/ruby-2.0.0-p247/gems/mechanize-2.7.3/lib/mechanize.rb:440:in get' from /usr/local/rvm/gems/ruby-2.0.0-p247/gems/linkedin-scraper-0.1.3/lib/linkedin-scraper/profile.rb:20:ininitialize' from /usr/local/rvm/gems/ruby-2.0.0-p247/gems/linkedin-scraper-0.1.3/bin/linkedin-scraper:4:in new' from /usr/local/rvm/gems/ruby-2.0.0-p247/gems/linkedin-scraper-0.1.3/bin/linkedin-scraper:4:in<top (required)>' from /usr/local/rvm/gems/ruby-2.0.0-p247/bin/linkedin-scraper:23:in load' from /usr/local/rvm/gems/ruby-2.0.0-p247/bin/linkedin-scraper:23:in

'

kirstywilliams commented 9 years ago

This happens when LinkedIn has blocked the IP address for too much traffic. You might want to limit the amount of calls you make to the site from a single IP address.

yatish27 commented 9 years ago

Limit the call with max of 4 requests per second. Do not exceed more than that