search

j-andrews7 / kenpompy

A simple yet comprehensive web scraper for kenpom.com.
https://kenpompy.readthedocs.io/en/latest/?badge=latest
GNU General Public License v3.0
70 stars 21 forks source link

CI/CD: Better handling for Cloudflare error 520 #74

Closed esqew closed 9 months ago

esqew commented 10 months ago

Starting to see 520 errors being thrown more frequently by the Cloudflare service that's now powering KenPom.com as of last season. Example

While very annoying, this is not by any means an essential issue to tackle since manually re-running failed jobs is a piece of cake. However it would be very nice to have a more automated way of handling when these errors occur in CI/CD.

esqew commented 10 months ago

An update - I'm considering adding flaky as a dependency to automatically re-test failures on 520 errors as it seems this error is becoming more prevalent.

esqew commented 9 months ago

This hasn't cropped up again in over a month, so I'm closing this assuming whatever the root cause was on KenPom's end has since been resolved.