As a user, if I provide a list of features to the WRES and ask it to handle the crosswalk, then I don't want it to fail if some of those features don't crosswalk successfully

[epag]

---

Author Name: Hank (Hank) Original Redmine Issue: 89176, https://vlab.noaa.gov/redmine/issues/89176 Original Date: 2021-03-10

---

This is related to #89118 and is intended to start as a discussion. If we decide to not allow this, then the ticket will be rejected.

For example, consider the declaration in #89118-1; it specifies a list of features by @left@ source, USGS gauge id, and asks that the WRES use the WRDS location service to fill in the features for the @right@ source, NWM reach id. I'm guessing most users at RFCs know their AHPS location ids (HB5) and paired USGS gauge ids, or know how to find them, but not necessarily the NWM reach ids that would crosswalk to their AHPS ids. Hence, they may want to list features by AHPS id or USGS gauge id, but ask that WRES use the WRDS location service to identify the NWM reach id. However, they aren't users of WRDS services and likely won't know which features are crosswalked, and so they are likely to list some, or many, features that cannot be crosswalked using WRDS.

Furthermore, they are likely to use copy-and-paste to copy that list into declarations, meaning that it would be better if they could build a single list of features to evaluate instead of having to curate multiple lists depending on the forecast model evaluated.

Given all of those suppositions, do we want the WRES to halt when a feature crosswalk cannot be completed for a listed @feature@, or merely issue warnings, which would then be exposed through improved logging (can't recall the ticket # for exposing fewer log messages to users)? Currently, the WRES errors out, either providing a list of features that failed to crosswalk (logging example in #89118-3) or not (logging example in #89118-1).

I would argue that warnings are better, since it would tell them which features could not be crosswalked, while still allowing them to curate a +single+ list of features, better facilitating a copy-and-paste approach to their evaluations.

This ticket can be resolved once we've identified that it should just be warnings and modify the code accordingly. This ticket can be rejected if we decided to leave it as an error.

Hank

[epag]

---

Original Redmine Comment Author Name: James (James) Original Date: 2021-03-10T14:13:06Z

---

I agree, especially in light of the discussion in #89113. One of the frictions with evaluating N features is when M of N features could succeed, but all fail because N-M fail. I think we need more cases that are in the "no data" bucket where other features can succeed (edit: I mean by way of analogy, not that we call more instances "no data" when they aren't), but we also want to fail early when an error means that all features will fail.

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2021-03-11T17:05:20Z

---

From meeting...

The idea was, if a user asked for something that couldn't be done, we should stop and tell them. If they ask for feature AAAA to be crosswalked and it fails, then isn't that a problem?

In this case, I (trying to think like NWRFC) attempted a "shotgun" approach to finding the features to evaluate. I believe that may be how they use it. Some features could not be crosswalked, so the WRES failed. Do we want it to fail?

Chris wondered what would happen if a script or other thingy generated a declaration with features. The user wouldn't know the list of features. The other thingy may not know which will crosswalk. So it shotguns it and WRES fails. Good or bad?

James compared to data, where if a feature is not in the data, we tend to be lenient. In this case, we are before the data; its just attempting to crosswalk and find the NWM reach id for a USGS gauge location. Do we want it to fail?

General agreement is to be more lenient. Omit features where a crosswalk failed; +log a warning with features that don't crosswalk+. Error out if that results in no features (which I think it already will).

Leaving in the backlog, for now, but don't let this slip too long since the RFCs will likely need this once they come on board.

Hank