Bunsly / JobSpy

Jobs scraper library for LinkedIn, Indeed, Glassdoor & ZipRecruiter
https://usejobspy.com
MIT License
555 stars 109 forks source link

Added a new example that incredibly speeds up the scraping process utilizing threading #146

Closed ebhechmer closed 1 month ago

ebhechmer commented 1 month ago

I used threading within my own implementations, so I figured I would add in an example similar to what I've done.

cullenwatson commented 1 month ago

the offset in Indeed just skips x pages but actually visits the pages. I'd recommend no offset and just change your search. No delay is needed as well. For a 500 results all you need is one thread. For 50,000 results, yes you should use threads.