Retry attachment downloads to tackle server outages or bad files

ropensci / ruODK

ruODK: An R Client for the ODK Central API

https://docs.ropensci.org/ruODK/

GNU General Public License v3.0

42 stars 13 forks source link

Retry attachment downloads to tackle server outages or bad files #48

Closed florianm closed 4 years ago

florianm commented 4 years ago

Feature

ruODK fails attachment downloads on incomplete files (bad uploads) or server hiccups. Retrying the download fixes the problem. Deleting the locally downloaded bad attachment file (76kB "photos") re-creates the problem. This problem shouldn't happen but has been observed with real-world data (WA turtle nesting census, 30k records/season). Restarting downloads of all the data sucks with large datasets.

We should make ruODK retry a download on failure (verbosely, and with opt-out param).

florianm commented 4 years ago

httr::RETRY("VERB", ...) provided the desired improvement. All functions using httr::VERB are rewritten to use httr::RETRY() and tested. On failure (three attempts as per defaults), ruODK returns the error messages from the third failed attempt, so user-facing behaviour is not changed.

This comes just in time for https://github.com/ropensci/software-review/issues/335.