Laughing-Man-Studios / Ski-Resort-Dashboard

A small dashboard to display some statistics about the local ski resorts I visit
MIT License
0 stars 0 forks source link

Setup Daily Scraping in Fly.io #63

Closed Rogibb111 closed 2 years ago

Rogibb111 commented 2 years ago

Fly.io doesn't have a scheduled job runner like Heroku does. In my Heroku app, I using a scheduler that would spin up quick computation instance and send a curl request to my app that would kick off the data scraping flow. Since I don't have a scheduler anymore, instead I'm going to use Fly.io's built in endpoint checker to do it instead. I will basically be copying my RobsAdventures Fly.io app approach. However, the Fly.io endpoint checker seem to run at much smaller time intervals than the twice a day that I want. I am going to have to add code that checks if I am within the time window for kicking off the scraping job. The time window will be the length of the interval, with my desired time directly in the middle of the window. This will ensure that job runs at least once, but no more than that.