search

hytest-org / hytest

https://hytest-org.github.io/hytest/
22 stars 11 forks source link

Research API call for SNOTEL #42

Closed alaws-USGS closed 1 year ago

alaws-USGS commented 2 years ago

Sydney: Emailed Gus (angus.goodbody@usda.gov) about SWE data access via S3. Right now, the interface to download is by state, or by individual station...

Got in touch with Beau Uriona at the National Water and Climate Center.

He said, "We do not host the data in such a manner [in an online bucket], however we have several simple methods that might be better than what you refer to as "download by individual gage or state". I find often it's best to set up a 30 min meeting to go over your data needs and go from there, we have so many ways to get our data it really depends on your ETL workflow - we may find one works better than another. "

I am asking if he can meet tomorrow potentially, his email slipped through the cracks this week (came in on Monday .. face palm)

edit: we are meeting tomorrow for 30 min!

Met with Beau, very cool, he is willing to help with sharing code he has, ETLs, questions etc. He has created this api as a hobby that we might be able to use. I am unsure of the longevity of this but it might be helpful.

The data live like this currently (in .json files, one per site): https://www.nrcs.usda.gov/Internet/WCIS/sitedata/DAILY/WTEQ/

We can trim the time period of interest in our wrapper, or Beau's if he can share data, in the files there is info on the period of record: "beginDate": "1996-09-01 00:00:00" "endDate": "2022-08-31 00:00:00"

All stations have varying start/end times. We are interested in 1979-10-01 through 2021-09-30 (USGS water year is Oct 1 to Sep 30).

edit: another note is that SNOTEL data can change over time with bias correction, etc. Longterm, we may want to think about frequently updating/pulling data.

sfoks commented 1 year ago

This is the API that Beau created as a hobby, https://api.snowdata.info/

alaws-USGS commented 1 year ago

@sfoks thank Beau for his hobby :)

rviger-usgs commented 1 year ago

you all are probably way ahead of me but want to ask if the NHGF crowd know about this? They might want to include this API in their catalog entry re: SNOTEL.