Migrate scraper job to the cloud

karkir0003 / GT-On-Campus-Jobs-Notifier

GT On-Campus Jobs scrapes job postings on https://studentcenter.gatech.edu/campus-jobs and sends email notifications to users as soon as a new posting is detected. Check Readme for setting up this application locally on your machine.

https://gtoncampusjobs.com

MIT License

1 stars 0 forks source link

Migrate scraper job to the cloud #4

Open karkir0003 opened 4 months ago

karkir0003 commented 4 months ago

We currently have a scraper that scrapes job postings from GT On Campus Job websites in a manually managed cron-job. Let's find a way to use Google Cloud Platform to host this scraper job on the cloud

karkir0003 commented 4 months ago

Factors to assess (including but not limited to):

Does GCP have a way to run a function that's relatively lightweight at a scheduled time?
What would the cost be from GCP to run such a function
What's the work involved in doing that? How can we test that the scraper job works?