jameswilburlewis
commented
6 months ago This should be mostly fixed now. We maintain a set of index URLs that had problems being retrieved, so we don't request them again in the same download() call. And for failed requests, we now pause for 2 seconds before making another request -- this may help with automated IP blacklisting on the ERG data server.
See this issue I just opened re: ERG: https://github.com/spedas/pyspedas/issues/745
It is possible that pyspedas.download() may be doing too many retries, in too short a timespan, if it encounters a server error when trying to download a directory listing. If that is happening, we may want to delay a short time (a few seconds?) between retries, to avoid triggering rate-limiting IP blocks at the data provider's firewall.