search

bmcgaughey1 / USGSlidar

A collection of functions to browse lidar data collections, query collections for coverage for specific locations, and retrieve data covering locations.
Creative Commons Zero v1.0 Universal
5 stars 1 forks source link

Add error checking to fetchFile() #11

Closed bmcgaughey1 closed 2 years ago

bmcgaughey1 commented 2 years ago

If a download in a list fails, things stop. Add tryCatch logic to trap the failed download when the URL cannot be opened. I think this is happening due to limits on the USGS server (limited requests or traffic from the same IP address) but i don't know for sure. If the error is trapped, it would be easy to re-attempt downloads for failed files. As the code is not, things stop when a file fails and there is no return from fetchFiles().

bmcgaughey1 commented 2 years ago

I added the tryCatch handling so a failed file returns a value of 1. This allows you (provided you save the return values) to determine which files in a list failed and, potentially, try to fetch them again. With the USGS public server, I see failed downloads somewhat frequently. I seem to remember reading somewhere that the server starts denying requests (or returning errors) when the load reaches some threshold. the behavior isn't necessarily related to requests coming from a single IP address but to all traffic on the server. However, I can't find the source for this information so it may be completely wrong.