search

spinlud / py-linkedin-jobs-scraper

MIT License
307 stars 84 forks source link

Skip promoted jobs? #46

Open vasup86 opened 1 year ago

vasup86 commented 1 year ago

Hello,

Is there a way to skip scraping promoted jobs? The promoted jobs are entirely irrelevant to the title of the position I'm searching for. It would be nice if there is a way to skip them. Thank you.

spinlud commented 1 year ago

Hi, just added skip_promoted_jobs query option. Give it a go!

DeonEmyr commented 1 year ago

Hi, when I run my query it usually full of Promoted Jobs (over 80%). Linkedin will usually place 100's of Promoted Jobs as the will be entire search results just full of promoted jobs.

When running the scraper may I ask if we can stop the scrape when the 25 jobs are all promoted and skipped. Otherwise the scraper will continue to run indefinitely.

spinlud commented 1 year ago

@DeonEmyr you can collect from metrics how many jobs have been skipped and act accordingly