search

workingonthejob / MTGO-Results-Scraper

Scraper for the weekly MTGO results.
0 stars 0 forks source link

Only one instance of script run at a time #19

Closed workingonthejob closed 1 year ago

workingonthejob commented 1 year ago

I noticed that the application can get into a state where once the Imgur API limit is reached and while it's waiting for the API limit to pass subsequent cron jobs of the same script can be run which can cause issues. One issue that I noticed was that it was uploading screenshots from the same event several times.

One possible option is to create a shell script that checks ps -aux for instances of whatever script it's trying to run and then if there is already one running instance then do nothing.

workingonthejob commented 1 year ago

I currently just have the cron running for the results page scraper set to run every hour instead of every 10 minutes so it can finish the upload finishes before the next time it runs.

workingonthejob commented 1 year ago

Fix d6be279dac95ab572324ba57b9f2fe41d4d57891

Paths are hard coded in the batch files. I think this can only be fixed in the provisioning issue in order to make the paths more confiurable/dynamic.