joeyism / linkedin_scraper

A library that scrapes Linkedin for user data
GNU General Public License v3.0
2.08k stars 575 forks source link

AttributeError: 'WebElement' object has no attribute 'find_element_by_class_name' #136

Closed danielhangan closed 1 year ago

danielhangan commented 2 years ago
Traceback (most recent call last):
  File "/Users/hangandaniel/Documents/interlinked-se/data/scrapers/lkd.py", line 14, in <module>
    person = Person("https://www.linkedin.com/in/danielhangan", driver=driver)
  File "/Users/hangandaniel/Library/Python/3.9/lib/python/site-packages/linkedin_scraper/person.py", line 62, in __init__
    self.scrape(close_on_complete)
  File "/Users/hangandaniel/Library/Python/3.9/lib/python/site-packages/linkedin_scraper/person.py", line 87, in scrape
    self.scrape_logged_in(close_on_complete=close_on_complete)
  File "/Users/hangandaniel/Library/Python/3.9/lib/python/site-packages/linkedin_scraper/person.py", line 116, in scrape_logged_in
    self.name = root.find_element_by_class_name(selectors.NAME).text.strip()
AttributeError: 'WebElement' object has no attribute 'find_element_by_class_name'

My code:

from selenium import webdriver
import chromedriver_binary
from linkedin_scraper import Person, actions

driver = webdriver.Chrome()

if __name__ == "__main__":
    email = "***"
    password = "***"
    actions.login(
        driver, email, password
    )  # if email and password isnt given, it'll prompt in terminal
    person = Person("https://www.linkedin.com/in/danielhangan", driver=driver)
dmehrotra commented 1 year ago

I'm having a similar issue. Any fixes?

joeyism commented 1 year ago

It may be a problem with login. Can you run

from selenium import webdriver
import chromedriver_binary
from linkedin_scraper import Person, actions

driver = webdriver.Chrome()

if __name__ == "__main__":
    email = "***"
    password = "***"
    actions.login(
        driver, email, password
    )  # if email and password isnt given, it'll prompt in terminal
    print("Press enter after you've successfully logged in")
    person = Person("https://www.linkedin.com/in/danielhangan", driver=driver)

and ensure that you're fully logged in when the chrome driver opens?

danielhangan commented 1 year ago

@joeyism I can see how the scraper is logging in and going to the person page. But it throws the error while loading the person's page, after it logged in.

Is there a time.sleep between load_page and and find_element_by_class_name?

abdou0102 commented 1 year ago

same problem any solution plz

joeyism commented 1 year ago

@joeyism I can see how the scraper is logging in and going to the person page. But it throws the error while loading the person's page, after it logged in.

Is there a time.sleep between load_page and and find_element_by_class_name?

Yea, it should wait until the top card is loaded.

See

jnzhnd commented 1 year ago

This is probably an issue with selenium - see this stackoverflow thread. Until this is fixed, you can solve it by downgrading selenium.

joeyism commented 1 year ago

For references, I have

selenium==3.141.0

and on chrome side

'browserVersion': '107.0.5304.110'
'chromedriverVersion': '107.0.5304.18

You can check your drivers and stuff by running

from selenium import webdriver
driver = webdriver.Chrome()
print(driver.capabilities)
RogeMetz commented 1 year ago

I'm facing the same error here:

Exception has occurred: AttributeError 'WebElement' object has no attribute 'find_element_by_class_name' File "C:\Users\rogem\Desktop\Project 02\LinkedIn Web Scraper.py", line 8, in person = Person("https://www.linkedin.com/in/rogemetz", driver=driver)

Any solution?

danielhangan commented 1 year ago

@RogeMetz just downgrade selenium version. It worked for me.

Hrovatin commented 1 year ago

Are there any plans on making this package run with never version of selenium? I think the change in retrieving elements has been there for a while.

teddsords commented 1 year ago

I had the same issue and by downgrading Selenium, it works fine.

popekabu commented 1 year ago

fixed mine by searching every word in the project having the keyword "find_element_by_class_name" with "find_element". It seems the former is no longer used, probably deprecated

joeyism commented 1 year ago

This should be fixed now