weather-gov / weather.gov

weather.gov 2.0
Other
315 stars 5 forks source link

Retry downloading corrupted zip files #1176

Closed greg-does-weather closed 3 weeks ago

greg-does-weather commented 3 weeks ago

What does this PR do? 🛠️

Use the unzip -t command to check if a zip file is valid before attempting to unzip it. If it's not, try downloading it again.

What does the reviewer need to know? 🤔

To test it locally, I added a return statement after line 46 of load-shapefile.js so the script wouldn't attempt to actually load anything into the database, updated the metadata version to 2 in sources/counties.js, and then invalidated my counties zip file by running truncate -s -50k spatial-data/c_05mr24.zip.

Then I ran the script, saw it identify the corrupted zip, and watched as it downloaded it afresh.