Pathway to Finalizing VBET 1.0 and running for 2/3 of US

[joewheaton]

@philipbaileynar and @joewheaton had a good strategy discussion on wrapping up VBET development. This is a reminder task list for @joewheaton to track (and FYI for @shelbysawyer and @lauren-herbine) for laying out the pathway and timeframes. Our goals are to:

1. Wrap up and complete planned algorithm changes early in the week of 12/12 (hopefully with 0.8.5 or 0.8.6 (shelving other algorithm considerations until a v1.1.x with a different SOW in future years)
2. Make final verification testing and (if necessary) calibration changes in the 0.9.x series preferably before Christmas
3. Approve a v1.0.x release and queue up runs of entire Mississippi and rest of Western US for January 1, 2023.

We think this is realistic, and that gives way to then have the riverscape DGOs and IGOs for about 1900 HUC 8s. We can then shift our Production Grade Network Tool focus to the hybrid Riverscape/Channel Network models @jtgilbert is refining (Anthro, BRAT, RCAT) and GNAT and Confinement that @KellyMWhitehead and @lauren-herbine are working on. If our estimates are correct, that will be something like:

• 93 million IGOs (@ 100 m spacing) over roughly 930,000 kilometers of riverscape (corresponding to roughly 5.5 million kilometers of NHD stream network).
• On average 40K to 50K IGOs per HUC 8

Status as of 12/9/22

VBET has undergone major development refinements and improvements as of late thanks to awesome work by @KellyMWhitehead, @MattReimer and @philipbaileynar. From September to present we have advanced from 0.5.x through to 0.8.x. We have a successful 0.8.3 run as of 12/7/22 (see DEC07 warehouse tag) and for 0.8 change log improvements see here.

Agreed Plan of Attack

This is less from the perspective of @KellyMWhitehead and @philipbaileynar and more from the perspective of what @joewheaton and @shelbysawyer need to provide them with to achieve above.

Algorithm Development / Refinement Phase

• [ ] 1. Wait until 0.8.4 runs are complete See #673
• [ ] 2. Confirm the fixes that were made up until 0.8.4 (minus the TauDEM issue) are producing desired impacts. i.e., that:
  • [ ] 2a. changing input parameter to use processing_extent layer instead of huc8 to match actual watershed boundary helped with a few of the dangling channel area inclusions outside the HUC.
  • [x] 2b. implementing raster cleaning to remove "vbet ribbons" (#671) removed the erroneous ribbons bleeding out of tributaries along mainstem valley bottom margins, and fixed the broken centerlines (i.e. multiple segments) and corresponding odd DGO and IGO geometries.
• [x] 3. Report any feedback or bugs (hopefully Monday PM or Tuesday AM depending on when 0.8.4 re-run is complete)
• [x] 4. While @joewheaton is reviewing 0.8.4, @philipbaileynar and @KellyMWhitehead work on following four improvements for 0.8.5 (Tue or Wed release if goes smoothly):
  • [x] 4a. Change IGO spacing parameter from 50 m to 100 m - _This is excepted to not only cut processing time, but create better geometries for the larger parts of network and adequate resolution for the first order systems. It is still less than what Herve and Samuel have typically done (250m to 500 m) for DGOs).
  • [x] 4b. Actually incorporate the #672 fixes (i.e. re-run TauDEM projects for TWI upstream of VBET, as well as the temporary non zero (0 → 0.0000001) slope fix, and the dealing with using the -nc flag with areadinf.
  • [x] 4c. Include the fix of waterbody NoLevelpath (#656) extent (i.e. don't derive a custom extent raster for analysis, but run the NoLevelPath as the entire HUC extent. I think it will be a minor fix and make a big visible difference (i.e. eliminating the noData wholes in the composite rasters on reassembly).
  • [ ] 4d. Add the missing metrics requested in #661
  • [ ] Depending on timing, if any minor tweaks or bugs are found in 0.8.4 review, consider including in 0.8.5.
• [ ] 5. @joewheaton review 0.8.5 and confirm 4a → d fixes having desired impact. If yes, VBET algorithm fixes pause while verification and calibration begin. If minor fixes, get a 0.8.6.

There are a variety of ideas for algorithm improvements that might either help with later LiDAR processing, or make VBET have less NHD+HR dependencies. However, for now, we DO NOT need to pursue such changes. We need to live with the changes we made to get us to a 1.0 that allows us to deliver on A48413, A48904 and A51544 scope (not new speculative scope).

NOTE, 0.8.5 will need to be a DEBUG run, so for verification if we need to sleuth and figure out the culprit at levelpath we can.

---

Verification & Calibration Phase

As soon as 5 is complete, we embark on verification using @shelbysawyer database of validation points. We used 10% of her sample points to specify the initial transform curves. We are going to user her remaining 90% for verification. We (i.e. @shelbysawyer with @joewheaton help) are verifying the following:

• A. The coherence of overall valley bottom extent (what is and what is not Valley Bottom) output.
• B. The coherence and performance of inactive vs. active predictions.
• C. The coherence of who's valley bottom is it via levelpath segmentation.
• D. The ability of VBET to pay respect to tributary fans (i.e. exclude them from original levelpath inclusion).
• E. The coherence of all VBET metrics (i.e. right and wrong answers for right reasons)

Things that can change in the iterative verification & calibration phase

If verification of A-E reveals poor performance that could be improved with calibration, we will ask for one of the following types of changes and request a re-run (by default on 20 HUC 10s unless otherwise specified):

• Any of the nine TWI, Slope and/or HAND transform functions can be adjusted
• The window size of metric calculation (not for individual metrics here, but at least by streamsize 0, 1, 2)
• The IGO spacing (preferably uniformly, but if absolutely necessary by streamsize). It CANNOT go < 50 m (but could go back to 50 m), and it should only be in 100 m increments if getting coarser.
• The specifics of how a metric is calculated or stored (i.e. a few lines of code fix, not 100s) to get a metric performing like we want it to.

If a minor bug is found in process of verification/calibration, we report it and ask for a fix, which will trigger a new release (e.g. 0.8.7).

Goal is for @shelbysawyer to begin calibration by end of week 12/15 or 12/16.

Verification is expected to produce the following possible outcomes:

• Passes, we're good.
• Minor calibration requested, re-run with new parameters' and repeat verification.

Hopefully, after one or two iterations (or none) we confirm (verify) that performance of VBET is adequate and we

• [ ] Approve parametrization and request VBET 1.0 to be released and queued up to run for 2/3 of US.

Hopefully @joewheaton checks this box by Christmas or New Years.

@shelbysawyer please make sure you are happy with above. @philipbaileynar, this is less for you and more for us, but if I am missing or misunderstanding anything from our conversation, please let me know.

[philipbaileynar]

I am in the process of closing out the "VBET 2022" milestone. Now that we are clear of urgent VBET algorithm fixes we can start using milestones tied to VBET versions.

VBET 0.8.6 runs are on the staging warehouse for USU to review and start calibrating.

[joewheaton]

This issue is no longer current. I think everything actually go done, but closing as is.