search

DOI-USGS / dataretrieval-python

Python package for retrieving water data from USGS or the multi-agency Water Quality Portal
https://doi-usgs.github.io/dataretrieval-python/
Other
165 stars 41 forks source link

Datetime Records Missing if time code is not present #47

Closed SarkenD closed 1 year ago

SarkenD commented 2 years ago

Hello- It would be great if the retrieval code be modified to pull date and time columns using the get_record function. It appears that the current method provides a date record if date AND a time column is filled out. When you run the retrieval program, there are 176,000+ records where datetime is returned as null. The function should pull the date and time column separately. The user can concatenate these together if need be using pandas.

Example Below: Shows a water level value for date sampled of 1983-04-21. Unfortunately, the dataretrieval scripts pulls together a datetime column with a null value if time is missing.

image

image

please reach out to me if you need more info. Sdressler@usgs.gov

pinshuai commented 2 years ago

I encountered the same issue when downloading groundwater levels. Have you figured out how to fix this?

jjkennedy commented 1 year ago

Same problem here. I emailed the contact listed at https://pypi.org/project/dataretrieval/ to see if there was a work around and he just directed me to here to create an issue.

thodson-usgs commented 1 year ago

@elbeejay, We've handled the timestamp issues for some services, though not all. Seems like we need to think about a more general solution.

elbeejay commented 1 year ago

Thanks for noting and describing the issue all. I'll have a look next week and come up with a solution, @SarkenD's proposal in the first post looks reasonable to me so will just double check to make sure that doesn't break anything else before implementing. @thodson-usgs I'll make sure any changes don't break existing functionality, but will introduce deprecation warnings if/as necessary.