search

catalyst-cooperative / pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
https://catalyst.coop/pudl
MIT License
471 stars 108 forks source link

CEMS FTP download gets trapped in a loop #234

Closed karldw closed 5 years ago

karldw commented 5 years ago

Describe the bug The CEMS FTP site doesn't have data for October, November, or December of 2017. When the datastore code tries to download the file, it should error out after the second attempt to download those missing files (see the logic here, which I wrote), but that doesn't happen.

To Reproduce Run update_datastore.pywith a constants.data_years that includes 2017 for the EPA CEMS data.

Expected behavior Expected an error downloading the missing files. One fix would be to remove 2017 from the CEMS data_years until those files are available. But it would be nice if the FTP downloader worked as designed.

Desktop (please complete the following information):

karldw commented 5 years ago

I was wrong. This isn't happening; I'm just impatient.