joeyism / linkedin_scraper

A library that scrapes Linkedin for user data
GNU General Public License v3.0
1.86k stars 527 forks source link

You are not logged in! #100

Open AKazlauskaite opened 3 years ago

AKazlauskaite commented 3 years ago

When I run my code, I get an exception "you are not logged in! please verify the capcha then press any key to continue...". But Im logged in and dont receive any captcha. The exception occurs after "person = Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5", driver=driver)", because the required page is open.

from linkedin_scraper import Person, actions from selenium import webdriver driver = webdriver.Chrome()

email = "some-email@email.address" password = "password123" actions.login(driver, email, password) # if email and password isnt given, it'll prompt in terminal person = Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5", driver=driver) person.scrape(close_on_complete=False) print (person)

joeyism commented 3 years ago

Did you replace email and password with your own email and password?

chiefeno commented 3 years ago

Same issue, no captcha and it says that I'm not logged in. I replace the email and password

joeyism commented 3 years ago

Can you try it with 2.9.0? A lot of things are fixed in that version

admond1994 commented 2 years ago

Hi @joeyism , I'm using 2.9.0 and still got the same issue as above. Any resolve?

joeyism commented 2 years ago

@admond1994 Can you paste the code you are using? I'll try to reproduce it.

admond1994 commented 2 years ago

Hi @joeyism , below is the code that I'm using:

from linkedin_scraper import Person, actions
from selenium import webdriver

path_to_chromedriver = "path-to-my/chromedriver"
driver = webdriver.Chrome(path_to_chromedriver)

# login details
email = "some-email@email.address"
password = "password123"

# if email and password isn't given, it'll prompt in terminal
actions.login(driver, email, password)

profile = "https://www.linkedin.com/in/admond1994/"
person = Person(profile, driver=driver)
person.scrape()

Below is the error that I got:

Screenshot 2021-07-27 at 11 37 40 PM
bulmandu commented 2 years ago

I have same unresolved error!! ㅜㅜ Did you resolved? Here my code

def scrap_profile_linked(email, pw, url):
    options = webdriver.ChromeOptions()
    options.add_argument("headless")
    driver = webdriver.Chrome('util/chromedriver', options=options)
    time.sleep(3)
    print("log.....로그인 중")
    try:
        actions.login(driver, email, pw)
        time.sleep(5)
    except Exception as e:
        print("log.....로그인 실패", e)
    try:
        person = Person(url, driver=driver)
    except Exception as e:
        print("log.....person 불러오기 실패", e)
    profile_info = person.scrape(close_on_complete=False)
    print(profile_info)
Mayank8085 commented 2 years ago

I am also facing the same issue if you find any solution please let me know

raf-kan commented 2 years ago

Hey everyone! I have the same issue as well, seems to be blocked by a captcha after going through the "connections" page. I have almost the exact same code as everyone above. Hope that helps.

donfouts commented 2 years ago

same problem too

a7med7asan15 commented 2 years ago

Same issue here, I'm not sure if resolved or not but sometimes linkedin sends an email with a number that you should write it in the field

LanikaiLi commented 2 years ago

same problem, too, is it because LinkedIn is protecting itself from being scrapped? Did any of you guys solved it?

jamesdeluk commented 1 year ago

Did anyone find a solution to this? I'm still getting this. These is no captcha - in fact, it's not even on the login page: image

RakuJa commented 1 year ago

I've found a possible solution!

Explanation

The problem is probably related to bot behavior and an internal bug of the library.

I've used this simple code snippet:

actions.login(driver, email, password)
person = Person(linkedin_url=f"https://www.linkedin.com/in/{profile_to_scrape}", driver=driver)

That raises this error:

image

At this point I thought that this problem is encountered probably because immediately after login the driver tries to load a page and requesting pages too fast is a bot behavior.

To avoid getting black listed (and logged out with the captcha error) I added a simple sleep. I tried with 0.1 and 0.2 seconds and it failed.

With 0.3 it worked but it loaded the page as a "non authenticated" user example

Finally with 0.4 onward it worked.

So the big bug here is that there is not enough delay between requests and LinkedIn automatically flags your client as a bot and redirects you away.

SOLUTION

Add some delay between the login action and the scraping. Obviously a sleep delay will differ, as a fast computer may have little to no delay between the login and scrape operation while a slower pc may already have enough delay to avoid getting flagged as a bot

actions.login(driver, email, password)
time.sleep(0.5)
person = Person(linkedin_url=f"https://www.linkedin.com/in/{profile_to_scrape}", driver=driver)