search

PaulMcInnis / JobFunnel

Scrape job websites into a single spreadsheet with no duplicates.
MIT License
1.78k stars 210 forks source link

More Sites to Scrape #52

Closed rpmullig closed 2 years ago

rpmullig commented 4 years ago

I attempted to adjust the 'providers' in the settings.yaml, but I found a few that raised errors. The following would be great additions to impact the tool:

PaulMcInnis commented 3 years ago

Just adding to this list here, so it is captured in the right place: https://news.ycombinator.com/jobs

PaulMcInnis commented 3 years ago

This seems like a decent source as well: https://remote.co/remote-jobs/

rpmullig commented 3 years ago

Before moving into development, I worked in Finance and found many venture capital firms that had websites with listings of jobs for their investments. I could find a few of the main ones and post here.

apontejosea commented 3 years ago

Is there any thoughts on inverting the dependency and make the scrapers pluggable? I'm thinking there is opportunity to define a clear interface/abstract class for people to implement their own scrapers. Later on, we could use the entrypoint mechanism to enable people to implement plugins in a separate package. In that way, the main JobFunnel framework can become more stable and anyone could create and maintain their own plugins as necessary as Python packages. Once in pypi, a requirements file could eventually look something like:

jobfunnel jobfunnel-linkedin jobfunnel-monster ...

PaulMcInnis commented 3 years ago

@josian I'd be interested to see this idea with a more fleshed-out example architecture, It might be a good idea for maintainability and as the bloat grows around special cases.

I could see that fitting into the current ABC design where we are essentially building Base objects and then specializing per-locale, per-provider.

If you're interested in taking this further, a design for the class stubs would be a good place to start the discussion.

PaulMcInnis commented 2 years ago

135 indeed India