search

PaulMcInnis / JobFunnel

Scrape job websites into a single spreadsheet with no duplicates.
MIT License
1.85k stars 215 forks source link

Added glassdoor. Some @TODO still need to be fulfilled. #5

Closed studentbrad closed 5 years ago

studentbrad commented 5 years ago

I added glassdoor.ca to JobPy. Some @TODO still need to be fulfilled. I will complete those before a merge to master.

I added a version number because I thought it was appropriate with the number of changes being made. I added /tools due to the similarities between job search sites, mainly the "relative post age" calculation.

I have decided to extract the "blurb" for monster and glassdoor from the job link for each job as it is not visible from the job soup. This will require a larger run time but I believe it is important because indeed job postings all include a "blurb".

studentbrad commented 5 years ago

I believe that this is a good time to go back and make corrections and optimizations to existing job scrapes. I have learned a lot about how sites scan for bots and malicious software. I had to implement a workaround due to the site protection with glassdoor. I believe I should be using the same methods as I do with glassdoor with the other sites (monster and indeed).

PaulMcInnis commented 5 years ago

This is awesome! 🥇