joeyism / linkedin_scraper

A library that scrapes Linkedin for user data
GNU General Public License v3.0
2.01k stars 560 forks source link

Profile scraping error on get_experiences #179

Open buliojarros opened 1 year ago

buliojarros commented 1 year ago

Trying to scrape a profile with the following code:

from linkedin_scraper import Person, actions from selenium import webdriver

driver = webdriver.Chrome()
email = "xxx@gmail.com"
password = "xxx"
actions.login(driver, email, password)
person = Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5", driver=driver, scrape=False)

person.scrape()

i get the following error

% python3 -m testscrape.py Traceback (most recent call last): File "", line 189, in _run_module_as_main File "", line 112, in _get_module_details File "/Users/julio/Documents/libri/testscrape.py", line 15, in person.scrape() File "/opt/homebrew/lib/python3.11/site-packages/linkedin_scraper/person.py", line 89, in scrape self.scrape_logged_in(close_on_complete=close_on_complete) File "/opt/homebrew/lib/python3.11/site-packages/linkedin_scraper/person.py", line 285, in scrape_logged_in self.get_experiences() File "/opt/homebrew/lib/python3.11/site-packages/linkedin_scraper/person.py", line 131, in get_experiences position_title = outer_positions[0].find_element(By.TAG_NAME,"span").find_element(By.TAG_NAME,"span").text ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/julio/Library/Python/3.11/lib/python/site-packages/selenium/webdriver/remote/webelement.py", line 425, in find_element return self._execute(Command.FIND_CHILD_ELEMENT, {"using": by, "value": value})["value"] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/julio/Library/Python/3.11/lib/python/site-packages/selenium/webdriver/remote/webelement.py", line 403, in _execute return self._parent.execute(command, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/julio/Library/Python/3.11/lib/python/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute self.error_handler.check_response(response) File "/Users/julio/Library/Python/3.11/lib/python/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"tag name","selector":"span"} (Session info: chrome=114.0.5735.198)

I've already changed my selenium version to 4.2.0

yannjor commented 1 year ago

Duplicate of #173. This pr should fix it #180.

joeyism commented 1 year ago

Pr #180 is merged. Please try again