pipeline for river network

[aappling-usgs]

non-MVP but woudl be cool

[dblodgett-usgs]

Here's a concept to consider. I just used the NLDI to get the upstream mainstem and downstream mainstem for each gage. For the sake of seeing what it looks like, I just plotted the upstream in red and downstream in blue.

I think this could be a good and simple way to demonstrate how the gages relate to the drainage network. It doesn't attempt to represent all the hydrography, but does show how the gages relate to a broader hydrologic network.

Thoughts?

[aappling-usgs]

I sorta love it - it's definitely the right density and shows how the gages are or are not connected to one another and to the landscape.

In case this wasn't already your intent, I think we should use just one color for the final hydrography lines, or at least not make an effort to distinguish visually between upstream and downstream. It's great for our discussion but I think would distract an uninitiated audience.

At the same time that I'm very happy with how this looks, I'm worried that somebody is going to think these are the major rivers (or our failed attempt to capture the major rivers). The disadvantage/challenge of a GIF-only viz is we don't get any text to support our communication, so it really really needs to stand alone.

As a next iteration, maybe we could try showing both:

1. in dark blue: the largest rivers in the network, regardless of whether they have our gages on them, and
2. in lighter blue: all rivers with gages on them, as in your prototype above but not re-plotting those that are both large and have gages

@dblodgett-usgs and others, what do you think - do you agree with my concern? do you think that my proposal above would get us out of trouble, or would it still be confusing?

[jordansread]

I agree that adding the up/downstream style difference adds a new dimension to the viz that may be confusing for the target audience. I like the proposal Alison has as a next step. Nice work Dave!

[dblodgett-usgs]

Yes, my intent with the color was for this issue only. I was imagining the same styling throughout for the viz.

I like your proposal, will work that up once I figure out a good way to seed "largest rivers in the network". I have an idea.

[jzwart]

This is awesome

[dblodgett-usgs]

This took a lot of doing, but I got it.

I went to the NHDPlus and pulled out COMIDs that are watershed outlets (terminal) as well as COMIDs that are the downstream end of tributaries that are not on the mainstem of the terminal rivers. The code depends on the national NHDPlus Flowlines layer -- a 7GB geo database. But I generate a 5MB rds with 14k potentially of interest outlet COMIDs. Could that be a source file for the project?

Regarding the data in the screen grab, I chose 15 sites at random from our set and the background rivers are order 7 and greater. We could also filter the background set to total drainage area but having played around with it, it doesn't work as well as order.

Thoughts?

[jordansread]

wow, that's a heavy dataset to depend on. Quick q - we can't hit your cida.usgs.gov web services (NWC?) w/ a stream-order query to pull out the big rivers?

[dblodgett-usgs]

I tried that and failed. There were a couple different potential solutions like that and couldn't come up with anything.

But to be clear, this solution only depends on the NHDPlus to seed the potential set of catchments as starting points for the NLDI.

[jordansread]

I see, and you are proposing skipping over that by including the comIDs that are pre-filtered as a source file to the repo? (or on drive)

[dblodgett-usgs]

correct.

[aappling-usgs]

ok, i think you should navigate to https://drive.google.com/drive/u/1/folders/1GtaYM6F2pjzomH7yurvdzb_OwFizOy61, use browser to create a folder called in, and upload that "5MB rds with 14k potentially of interest outlet COMIDs" into that folder (1_fetch/in/river_network.rds or similar).

then go to 1_fetch.yml and add these lines:

  1_fetch/out/river_network.rds.ind:
    command: gd_confirm_posted(target_name)
  1_fetch/out/river_network.rds:
    command: gd_get('1_fetch/out/river_network.rds')

and test them out and see if they do something reasonable. e.g., call each of these lines in succession and watch your repo for files appearing:

scmake('1_fetch/out/river_network.rds.ind')
scmake('1_fetch/out/river_network.rds')

then write the rest of your pipeline so it follows from '1_fetch/out/river_network.rds.ind'

[aappling-usgs]

I think the screenshot above looks great density-wise. is light blue the major rivers and dark blue the major and/or gaged rivers? if so, could we swap it so dark blue is major rivers (gaged or not) and light blue is gaged non-major rivers? or even mess with width a little, make major rivers be thicker and non-major rivers thinner?

[dblodgett-usgs]

OK, as the code stands, and what I'll get checked in we have this:

major rivers on top in dark blue and gaged rivers in light blue.

I'm not happy with the distribution of major rivers. I have a better selection algorithm that I can get implemented, but it will be additive to what I have. No reason not to get this checked in now.

[aappling-usgs]

Neato! Yeah, check it in.

[dblodgett-usgs]

PR for this is inbound, but just to finish the thread with what I've come up with in the issue, here's were we are at.

This uses one threshold for rivers that terminate at the coast and another threshold for rivers that are inland tributaries. That allows us to increase density near the ocean independent of the density inland. The png attached uses the threshold that will be in my PR in a second. 1000 sqkm for coastal drainages and 40,000 sqkm for inland. It's extreme, but it is meant as a starting point to highlight the configuration potential. There is a further configuration option for the line-work simplification. I am using a 5km tolerance, which seems to look good without over simplifying the stream lines at the scale we are looking at right now. More detail about what I did in my PR to come.

[aappling-usgs]

I love that we're getting several gages apiece on some of these rivers - would be a cool way to see how far inland the storm is reaching, or how inland flood waves propagate downstream. I bet that for some vizzies it'd be cool to arrange the sparklines so that gages along a single river are shown together in upstream-downstream order, and/or to plot by farther-to-closer to the storm, or inland to coastal.