joeyism / linkedin_scraper

A library that scrapes Linkedin for user data
GNU General Public License v3.0
2.13k stars 584 forks source link

Trying to scrape emplyees of a company always returns a list of None's #96

Open aradzekler opened 3 years ago

aradzekler commented 3 years ago

No error or anything, just doesnt seem to work on employees, anyone succedded in scraping employees from company?

joeyism commented 3 years ago

Do you have code on how you're trying to do it?

aradzekler commented 3 years ago

same as the latest fork, the company line itself is:

company = Company(row['linkedin_profile_generic'], driver=self.driver,
    close_on_complete=False, get_employees=True)
input_df.at[0, 'employees'] = company.employees

outputs: [None, None]

joeyism commented 3 years ago

I ran this code

import os
from linkedin_scraper import Person, Company, actions
from selenium import webdriver
driver = webdriver.Chrome("./chromedriver")
actions.login(driver, os.getenv("LINKEDIN_USER"), os.getenv("LINKEDIN_PASSWORD"))
company = Company("https://www.linkedin.com/company/pinnacle-west-capital-corporation/", get_employees=True, driver=driver)
print(company)

then when I ran

len(company.employees)

I get Screenshot from 2021-04-21 20-03-14

aradzekler commented 3 years ago

I get a list the size of the company employees but the list contains only None's

joeyism commented 3 years ago

Screenshot from 2021-05-02 00-31-36

Some of them may be None if it can't scrape it, but there are definitely some there.

The details aren't filled out because this is scrapped just from the company's employees list, which doesn't contain much info. If you want to get more details, you'll have to loop over the employees and scrape each one.

ErlingLie commented 2 years ago

When I scrape each person separately I get a result like your image over there with all fields as empty lists. Any ideas how to fix this?