search

LorenzoLaCorte / internship-scraper

Engineering Internships Scraper, developed to scrape european tech internships.
14 stars 2 forks source link

Explore additional data sources #4

Open LorenzoLaCorte opened 1 year ago

LorenzoLaCorte commented 1 year ago

Explore additional data sources: expand our scraping capabilities beyond city-based data to include company-based data. This may involve scraping from company career sites or LinkedIn research filtered by specific companies.

C-Nubela commented 1 year ago

Hey Lorenzo,

Have you considered integrating an external API as a data provider? I work at Proxycurl, a B2B data provider, and we could definitely provide all of your company-based data.

We're way easier than trying to scrape the data yourself. Consider giving us a try.

LorenzoLaCorte commented 1 year ago

Explore Google Jobs query: It's based on your current position and it gives you a quite good querying method. So, I think we can:

  1. tune a good query
  2. For every city in cities.txt
    • change the curr position to {city}
    • scrape the results
LorenzoLaCorte commented 1 year ago

Explore the Simplify system, which is very good but is only for US and UK