User Add Additional Lines of Valley Bottom Evidence

[joewheaton]

Request

Is it possible to queue up and accept additional lines of valley bottom evidence. Examples that come to mind are:

• FEMA Flood Maps, which can include the:

  • "Floodway" as an additional line of evidence for active channel
  • "1% Annual Chance Flood Hazard" - Will include "active floodplain" but also a lot of stuff that does not regularly flood (i.e. flooding more than every 5 - 10 years)
  • "0.2% Annual Chancel Flood Hazard" - *Also not a bad proxy for "non-functional"
  • "Area with Reduced Risk due to Levee" - this should be "non-functional"

• User uploaded polygon their call...-

• Quaternary Geology map (anything mapped as Qa (Quaternary Alluvium) or a specific alluvial surface (not terraces) is pretty good:

• Wetland Maps (e.g. NWI - Wetlands within valley bottom areas are a pretty good indication of "active floodplain" and a good line of evidence of being active valley bottom.

• Fluvial Hazard Zone Maps - There are an increasing number of channel migration zone, freedom space and fluvial hazard zones that are helpful where they exist:

Feature

Initially a command line utility, to be able to add to riverscapes project additional lines of evidence, and have arguments for their weight and populating metadata tags (what, who, when).

[philipbaileynar]

I am extremely keen to implement this. Our methodology is easy to extend to additional evidence layers.

I have spent some time investigating sources for both geological and FEMA data. My original desire was to find systematic downloads like we have for NHD, NTD, NED and HAND, so that we can automate their incorporation during cloud processing. Unfortunately there is a growing trend for providers to make their data available through curated web mapping tools like those in your screenshots above. It's virtually impossible to find national or even regional layers for download. This is particularly true of high resolution data like the FEMA flood maps. But I was even disappointed that I couldn't find a single national geological raster of any resolution for download (like we do with Landfire). I find this hard to believe.

So there is some rather time consuming research into web mapping services needed. My knowledge of these is not what it could be. My understanding is that there are several types, the most common of which merely provides data for display (i.e. your web request provides bounding rectangle and you get back PNG tiles) that are unusable for analysis. There are other service types such as Web Feature Service (WFS) that do provide full vector data over the internet, but they are less common. Anyway... research is needed.

As you conclude above, we will need a method for the user to contribute evidence layers of various kinds. We will need to think about whether they prepare these layers themselves or add them to the VBET project together with some kind of transform function and weight to be used when combining with other layers.

[MattReimer]

I think the biggest concern here is the widespread formats and levels of quality that you find these layers in. We'll need some way to parametrize them and sanitize them so that they're usable by VBET. Additionally we'll need some way of entering an arbitrary number of evidence lines. This could get really tedious on the command line and maybe we should look at something like a config gile (think .fis file) to help us make sense of this and do repeatable runs.

[philipbaileynar]

@MattReimer I agree with the config file and variable number of evidence rasters. This request is messy but essential.

One comment about availability.... it's inevitable that VBET will need to handle user-defined evidence inputs; ad hoc layers for a single HUC. Or in other words, not all evidence rasters will exist everywhere. This is regardless of whether we figure out a consistent way to retrieve FEMA flood map data etc. Users are going to need a way to say... "I have this useful polygon for my valley that I want to influence the VBET algorithm".

VBET is going to transition from a tool that runs the same way everywhere (entirely automated in the cloud) to a tool that users interact with and configure (more like GCD). Users add layers, tweak parameters and then re-run. Also the delineation of valley bottom that the tool does now, is just the first of several operations planned under the umbrella of the VBET tool (see issues Riverscapes/riverscapes-tools#14, Riverscapes/vbet#7, Riverscapes/vbet#8). This is why I created the ticket to start the discussion about how we architect all this (#12).

[joewheaton]

I'm liking the discussion. A few thoughts on specific comments below. First, I'm generally thinking for all this scoping discussion we need to think about milestones of where we are at currently, where we'll get with BLM, and where we eventually want to and could go.

Thinking about VBET scoping into different Grade Tools.

In other words:

• Currently - we are talking about getting our Beta Production Grade Tool working correctly and outputs consumable from warehouses into RAVE. Any "features" or functions added should be command-line tools (maybe that we in lab run or NAR runs), that add inputs, intermediates and outputs to VBET riverscapes projects.
• Next Phase - Commercial-Grade - our current proposal seeks funding to make a "commercial-grade" VBET, which means that web interactivity and functionality is a must. This may be a place to add or allow through a web-interface access to the outputs of some of these new functions, but we will want to think carefully about which if any of these features are exposed to "web-users" as tools, versus exposed to "web-users" as views. Any "GIS-user" will be a RAVE user and they can see everything.
• An Important non-scoped (yet) Phase will be backfilling with a Professional-Grade GIS tool. This would be something like GCD. Like GUT, I think this would have a huge audience of "GIS-users", and it may be something we consider exposing many of the command line tools we're talking about adding to VBET in an easy to use interface.

Thoughts on Comments

Being clearer about audience

From @philipbaileynar

As you conclude above, we will need a method for the user to contribute evidence layers of various kinds. We will need to think about whether they prepare these layers themselves or add them to the VBET project together with some kind of transform function and weight to be used when combining with other layers.

I'd say that if we're talking about a command line functionality to support the Commerical-Grade, this is a really small audience of "Power GIS-Users" (like ourselves and lab) and we put the prepping burden largely on ourselves. By contrast, exposing these as tools to "Web-Users" is really scary unless what we're doing is just letting them choose whether to include existing curated optional inputs (e.g. FEMA maps). If we are serving "GIS-Users" through a professional-grade tool, then we invest in the same way we did with GCD to handle any idiot under the sun.

Handling different inputs - Depends on what grade-tool we're talking about

From @MattReimer

I think the biggest concern here is the widespread formats and levels of quality that you find these layers in. We'll need some way to parametrize them and sanitize them so that they're usable by VBET. Additionally we'll need some way of entering an arbitrary number of evidence lines. This could get really tedious on the command line and maybe we should look at something like a config gile (think .fis file) to help us make sense of this and do repeatable runs.

See thoughts above on what grade tool we're talking about targeting and also if just exposing access to known data type (or one government hosts or we host), then you can sanitize easily. I LOVE the idea of a config file for this. Very reasonable. Also the sort of thing a power user would do. If exposing this in web, we're talking about a much more restricted and sanitized set of options.

Handling Evidence of Variable Coverage and Quality

From @philipbaileynar

One comment about availability.... it's inevitable that VBET will need to handle user-defined evidence inputs; ad hoc layers for a single HUC. Or in other words, not all evidence rasters will exist everywhere. This is regardless of whether we figure out a consistent way to retrieve FEMA flood map data etc. Users are going to need a way to say... "I have this useful polygon for my valley that I want to influence the VBET algorithm".

Lets not confuse what VBET might provide commercially "out of the box" for free everywhere, with facilitating ownership. I think in a commerical grade tool, we can tackle the simplest and easiest of these (i.e. just letting them choose which of pre-formatting existing coverages we host) to use and combine in the way they want to and make huge gains on the user-ownership idea. Serving "GIS-Users" is the scariest, because they really can throw a lot of stupid things at you. We'll need to constrain that. Serving "Power-GIS Users" can be a very user beware experience.

[joewheaton]

After embarking on path for a while in VBET of more evidence is better (i.e. with TWI), I'm not sure this is a good idea. I'm going to close and shelf it. We can always dig it back up in future if need be.