search

joeyism / linkedin_scraper

A library that scrapes Linkedin for user data
GNU General Public License v3.0
1.86k stars 527 forks source link

Expandable Experiences and Education #80

Open lukaskoeni opened 3 years ago

lukaskoeni commented 3 years ago

Hi there,

I realized that when people on LinkedIn have more than a certain number of entries in their experience or education section, the scraper does not recognize the entries that only show up, when you click 'show more'.

To scrape all the available information, the scraper would need to detect the show more button and click it, before collecting the html elements.

Have not come around to implement those changes myself.

joeyism commented 3 years ago

@lukaskoeni Do you have an example of when this happens?

lukaskoeni commented 3 years ago

For this profile here, as an example:

https://www.linkedin.com/in/katina-kralevska-ph-d-a8574416/

Under Education there are 4 entries, but I only see the 4th, when I click on 'show more'. The Scraper only retrieves the first 3 entries.

joeyism commented 3 years ago

@lukaskoeni Try it with 2.7.4 and let me know if it works. It worked when I tested it, but I want to make sure it works for you too