search

hytest-org / hytest

https://hytest-org.github.io/hytest/
22 stars 13 forks source link

Create CONUS404 spatial aggregation notebook using gdptools #532

Open amsnyder opened 1 month ago

amsnyder commented 1 month ago

We would like to replace this notebook with one that goes through essentially the same process using gdptools.

I think we can use the same set of polygons used in the example - WBD HUC12s to start. And maybe we can aggregate the CONUS404 daily dataset from the OSN pod.

amsnyder commented 1 month ago

We have interest in aggregating to the GFv1.1 and NHDv2. Do you have an idea of the best way to access those sets of stream networks? Is it using NLDI?

amsnyder commented 1 month ago

@sfoks pointed out this could likely just be a copy of this use case: https://nhgf-catalog.dev-wma.chs.usgs.gov/usecases/3f74f920-c00f-4369-aecc-8f885cddda4f/

rmcd-mscb commented 1 month ago

@amsnyder and @sfoks Haha, yes that works but there are some ways to deal with the period of record, that is if you want to aggregate 40+ yrs, I break it up by year and have worked out some code to do that. Becomes especially important when doing CONUS scale fabrics.

amsnyder commented 1 month ago

Ok - I think the larger scale workflow would be good to demo because I think several groups want to do that.

ted80810 commented 1 month ago

We have interest in aggregating to the GFv1.1 and NHDv2. Do you have an idea of the best way to access those sets of stream networks? Is it using NLDI?

If you are looking for use cases for GFv1.1, the RIMBE workflow right now aggregates to GFv1.1 with gdptools. However, we use a function to grab the GFv1.1 zip file from ScienceBase because we need the HRU boundaries. We do have pyNHD integrated into the workflow for WBD - HUC12s as an option. Although, I am not sure with NHD version it is pulling.

amsnyder commented 1 month ago

We have interest in aggregating to the GFv1.1 and NHDv2. Do you have an idea of the best way to access those sets of stream networks? Is it using NLDI?

If you are looking for use cases for GFv1.1, the RIMBE workflow right now aggregates to GFv1.1 with gdptools. However, we use a function to grab the GFv1.1 zip file from ScienceBase because we need the HRU boundaries. We do have pyNHD integrated into the workflow for WBD - HUC12s as an option. Although, I am not sure with NHD version it is pulling.

Hmm ok that's good to know. I think I'd rather not pull from ScienceBase because sometimes it is offline and I'm also trying to avoid the pattern of everyone downloading a copy of the data to their computer to run a workflow. If there isn't an API or anything that we can pull GFv1.1 from, I could upload it to the OSN pod and then users could read the data from there directly. I'll work on getting that data uploaded there!