yatish27 / linkedin-scraper

Scrapes the public profile of the linkedin page
MIT License
553 stars 221 forks source link

Returns nil in production but fine in development #21

Closed mmahalwy closed 10 years ago

mmahalwy commented 10 years ago

Using Ruby 2.1.1 and Rails 4.0.3

Development, works great. Production, I get this

999 => for -- http://www.linkedin.com/in/mmahalwy

yatish27 commented 10 years ago

Can you paste you entire trace

mmahalwy commented 10 years ago

Working in rails console

Loading production environment (Rails 4.0.4) 2.1.1 :001 > @profile = Linkedin::Profile.get_profile( 'http://www.linkedin.com/in/mmahalwy') 999 => for -- http://www.linkedin.com/in/mmahalwy => nil

yatish27 commented 10 years ago

Works fyn for me on local machine

On Tue, Mar 18, 2014 at 10:59 PM, Mohamed El Mahallawy < notifications@github.com> wrote:

Working in rails console

Loading production environment (Rails 4.0.4) 2.1.1 :001 > @profile https://github.com/profile = Linkedin::Profile.get_profile( 'http://www.linkedin.com/in/mmahalwy')

999 => for -- http://www.linkedin.com/in/mmahalwy => nil

Reply to this email directly or view it on GitHubhttps://github.com/yatish27/linkedin-scraper/issues/21#issuecomment-37962418 .

Yats

mmahalwy commented 10 years ago

Local works great, but production doesn't work

yatish27 commented 10 years ago

Check if linkedin is not blocked On 18-Mar-2014, at 11:11 pm, Mohamed El Mahallawy notifications@github.com wrote:

Local works great, but production doesn't work

— Reply to this email directly or view it on GitHub.

mmahalwy commented 10 years ago

How do i do that?

carlotorniai commented 10 years ago

Probably Linkedin is blocking your prod machine IP. I had the same problem and I had to use tor to get my app to work.

mmahalwy commented 10 years ago

What's tor?

carlotorniai commented 10 years ago

https://www.torproject.org/

mmahalwy commented 10 years ago

@carlotorniai do you have an example to how you used it? Would i just do: res = Tor::HTTP.get(Linkedin::Profile.get_profile( 'http://www.linkedin.com/in/mmahalwy'))

carlotorniai commented 10 years ago

That should work basically you want to redirect the http results to tor. But I will go back to my code as soon as I have the time.

mmahalwy commented 10 years ago

Perfect, thanks Carlo!

mmahalwy commented 10 years ago

Tried Tor, not working well with new ruby?

mmahalwy commented 10 years ago

@yatish27 anyway to have it not blocked by linkedin?

mmahalwy commented 10 years ago

Just created a small server on heroku and returns nil, same problem as noted above (that was on Rackspace).

I am not seeing anything from mechanize. Not working in production maybe?

mmahalwy commented 10 years ago

Any update on this issue?

DerekChia commented 10 years ago

Check out this issue at Mechanize - https://github.com/sparklemotion/mechanize/issues/336

Seems like linkedin is blocking...

mmahalwy commented 10 years ago

Yep, had to proxy