search

fantasy-toolz / mlb-predictions

Predictions for the outcome of MLB games.
2 stars 0 forks source link

Improve robustness in draftkings.py #4

Closed michael-petersen closed 5 months ago

michael-petersen commented 5 months ago

draftings.py runs automatically every six hours, scraping present odds from DraftKings. The DraftKings interface appears to be updated in semi-real time, which means that a snapshot grab of the lines may get a garbled version of the webpage.

To circumvent this, it would make sense to add some robustness checks to the scraper, to try certain parts of the code a couple of times if an error is detected.

michael-petersen commented 5 months ago

Worse, there seems to now be some blocking of requests.get() calls. Not sure what is going on here, since I can still see the json file. Add an override option to put in the file manually, with ff62e60.

Hopefully this just sorts itself out, otherwise we'll have to test some new scraping strategies later.

michael-petersen commented 5 months ago

Two recent commits have helped: First, the relevant json from sportsbook.draftkings.com is now obtained through a wget call, set up in d2a6582.

Second, 66f1030 sets a 'failure mode' for O/U lines that don't exist for whatever reason.

Closing for now, since this does seem to have worked, but will reopen if more failures occur.