search

joeyism / linkedin_scraper

A library that scrapes Linkedin for user data
GNU General Public License v3.0
2.01k stars 560 forks source link

Timeout Error #206

Open parthbhalerao opened 9 months ago

parthbhalerao commented 9 months ago 
from linkedin_scraper import Person
from selenium import webdriver
driver = webdriver.Chrome()
actions.login(driver, email, password)
person = Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5", driver = driver, scrape=False)
person = person.scrape()
person

I am using the code above and not made a single change to the file person.py. However, I am running in this issue continuously. I am not sure why:

Output exceeds the size limit. Open the full output data in a text editor

TimeoutException Traceback (most recent call last) Cell In[27], line 6 4 actions.login(driver, email, password) 5 person = Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5", driver = driver, scrape=False) ----> 6 person = person.scrape() 7 person

File ~/venv/lib/python3.11/site-packages/linkedin_scraper/person.py:89, in Person.scrape(self, close_on_complete) 87 def scrape(self, close_on_complete=True): 88 if self.is_signed_in(): ---> 89 self.scrape_logged_in(close_on_complete=close_on_complete) 90 else: 91 print("you are not logged in!")

File ~/venv/lib/python3.11/site-packages/linkedin_scraper/person.py:259, in Person.scrape_logged_in(self, close_on_complete) 256 driver = self.driver 257 duration = None --> 259 root = WebDriverWait(driver, self.__WAIT_FOR_ELEMENT_TIMEOUT).until( 260 EC.presence_of_element_located( 261 ( 262 By.CLASS_NAME, 263 self.__TOP_CARD, 264 ) 265 ) ... 13 chromedriver 0x0000000104dc3af8 chromedriver + 4061944 14 chromedriver 0x0000000104dd42e4 chromedriver + 4129508 15 libsystem_pthread.dylib 0x0000000189443034 _pthread_start + 136 16 libsystem_pthread.dylib 0x000000018943de3c thread_start + 8

natdennett commented 9 months ago

I'm running into the very same issue, for what it's worth! I'm noticing that Selenium is taking forever to run, also. I'm going to try and speed up the Chrome driver first. It's worth noting that I can get it to work, it just takes forever!

parthbhalerao commented 9 months ago

I get the error within 23 seconds and the scraping isn't complete. There's something wrong with the scrap function.