search

DOI-USGS / dataretrieval-python

Python package for retrieving water data from USGS or the multi-agency Water Quality Portal
https://doi-usgs.github.io/dataretrieval-python/
Other
165 stars 41 forks source link

Data downloading issue with a specific gauging station #129

Closed pppxiyu closed 9 months ago

pppxiyu commented 10 months ago

I tried to download iv data from about 1k gauges using a for loop. All of the iterations was successful except for gauge '05427530'. Error: "KeyError: '72255_y'".

Codes:

nwis.get_record(sites='05427530', service='iv', start='2018-12-24', end='2023-12-24')
thodson-usgs commented 10 months ago

Interesting. I started looking into it but haven't gotten to the bottom of it yet. Gauges are operated by Water Science Centers that sometimes follow different conventions, which might be breaking something in dataretrieval. We already do opinionated things to compensate for such problems and I mainly want to ensure that whatever we introduce does more good than harm.

thodson-usgs commented 9 months ago

@pppxiyu, I think I've resolved the issue but, I'd like to test this on your loop. Would install the patch in a new or existing environment and rerun your code?

pip install git+https://github.com/DOI-USGS/dataretrieval-python@129-data-downloading-issue-with-a-specific-gauging-station

If you're using an existing environment, be sure to remove dataretrieval and reinstall when you're done.

thodson-usgs commented 9 months ago

Nope, this breaks multisite retrievals. Working on the fix

thodson-usgs commented 9 months ago

@pppxiyu, okay test it now if you would. As before:

pip install git+https://github.com/DOI-USGS/dataretrieval-python@129-data-downloading-issue-with-a-specific-gauging-station

and if you didn't create a new environment, you'll want to pip remove dataretrieval when you're done, then reinstall from pypi/conda to ensure your environment keeps track of any updates.