search

DOI-USGS / national-flow-observations

This repository pulls national flow data from NWIS
Other
4 stars 8 forks source link

Update for WY2021 #18

Open limnoliver opened 3 years ago

limnoliver commented 3 years ago

Per a conversation with Janet, let's plan on annually pulling these around Oct 15 to capture the last water year. One consideration is checking with @lindsayplatt to be sure the quarterly River Conditions visualization is complete so we don't modify the historical data mid-build of that visual.

padilla410 commented 2 years ago

I believe I am completing this task this year. How about I schedule an annual issue for this each year using github actions as part of my current body of work?

lindsayplatt commented 2 years ago

This should be completed thanks to code changes from #24 and #26, and the bulk of the build updates happened in #27. Big performance improvement for the data pulling component due to a switch to Tallgrass + change from rds to qs file formats (see #21). Took ~ 13 hours to build the entire pipeline on Tallgrass and get outputs onto S3.

lindsayplatt commented 1 year ago

Commenting here to say that I need these flow data for another project and would like to kick off a build on Tallgrass. Is there any reason I should not do that?