cmu-delphi / covidcast-indicators

Back end for producing indicators and loading them into the COVIDcast API.
https://cmu-delphi.github.io/delphi-epidata/api/covidcast.html
MIT License
12 stars 17 forks source link

Add a retry loop for google symptoms #2034

Open aysim319 opened 1 month ago

aysim319 commented 1 month ago

When bigquery api call fails with a 500 error code*; we fail without retrying. We should add a retry loop when error code is 500 or the error message is backendError

There's a known issue with python request that 500 code gets converted into 400 (https://stackoverflow.com/questions/41129805/why-is-google-bigquery-api-returning-a-400-instead-of-a-500-status-code)

and in the case that it returns a 400 error code we also need to check for the error message

Context

https://delphi-org.slack.com/archives/C01LZ3A2UMU/p1724504475248319

https://cloud.google.com/bigquery/docs/error-messages