epag
commented
1 month ago Original Redmine Comment Author Name: Hank (Hank) Original Date: 2021-11-24T16:45:56Z
Just quickly tested a code change to see what impact it would have.
If I construct the @ThresholdOuter@ instances for WRDS thresholds from rating curves so that they include text after the name indicating the source of the rating curve from which the threshold is derived, that is enough to ensure that, when the rating curve source is not declared by the user, all thresholds from all rating curves are evaluated (see #98837). For example:
01464000_2590137_TACN4_NWM_Medium_Range_Single_Valued_SAMPLE_SIZE.csv:FEATURE DESCRIPTION,EARLIEST ISSUE TIME,LATEST ISSUE TIME,EARLIEST VALID TIME,LATEST VALID TIME,EARLIEST LEAD TIME IN SECONDS [INSTANTANEOUS],LATEST LEAD TIME IN SECONDS [INSTANTANEOUS],SAMPLE SIZE All data,SAMPLE SIZE >= 1362.5 ft3/s (action from rating curve USGS Rating Depot),SAMPLE SIZE >= 1362.5 ft3/s (bankfull from rating curve USGS Rating Depot),SAMPLE SIZE >= 1363.0 ft3/s (action from rating curve NRLDB),SAMPLE SIZE >= 1363.0 ft3/s (bankfull from rating curve NRLDB),SAMPLE SIZE >= 2034.0 ft3/s (flood from rating curve NRLDB),SAMPLE SIZE >= 2034.0 ft3/s (minor from rating curve NRLDB),SAMPLE SIZE >= 2034.2 ft3/s (flood from rating curve USGS Rating Depot),SAMPLE SIZE >= 2034.2 ft3/s (minor from rating curve USGS Rating Depot),SAMPLE SIZE >= 2537.0 ft3/s (moderate from rating curve NRLDB),SAMPLE SIZE >= 2537.2 ft3/s (moderate from rating curve USGS Rating Depot),SAMPLE SIZE >= 3368.0 ft3/s (major from rating curve NRLDB),SAMPLE SIZE >= 3368.3 ft3/s (major from rating curve USGS Rating Depot)
If I further change the the key in a map that stores the thresholds to use the full label, I can obtain outputs for the raw thresholds as well as the rating curve thresholds when a user does not clarify what they want. For example:
[Hank@nwcal-wres-ti01 output_for_NJ_with_threshold_name_update_individual_features_raw_too]$ grep FEATURE 01464000_2590137_TACN4_NWM_Medium_Range_Single_Valued_SAMPLE_SIZE.csv
FEATURE DESCRIPTION,EARLIEST ISSUE TIME,LATEST ISSUE TIME,EARLIEST VALID TIME,LATEST VALID TIME,EARLIEST LEAD TIME IN SECONDS [INSTANTANEOUS],LATEST LEAD TIME IN SECONDS [INSTANTANEOUS],SAMPLE SIZE All data,SAMPLE SIZE >= 720.0 ft3/s (action),SAMPLE SIZE >= 1040.0 ft3/s (flood),SAMPLE SIZE >= 1362.5 ft3/s (action from rating curve USGS Rating Depot),SAMPLE SIZE >= 1362.5 ft3/s (bankfull from rating curve USGS Rating Depot),SAMPLE SIZE >= 1363.0 ft3/s (action from rating curve NRLDB),SAMPLE SIZE >= 1363.0 ft3/s (bankfull from rating curve NRLDB),SAMPLE SIZE >= 2034.0 ft3/s (flood from rating curve NRLDB),SAMPLE SIZE >= 2034.0 ft3/s (minor from rating curve NRLDB),SAMPLE SIZE >= 2034.2 ft3/s (flood from rating curve USGS Rating Depot),SAMPLE SIZE >= 2034.2 ft3/s (minor from rating curve USGS Rating Depot),SAMPLE SIZE >= 2537.0 ft3/s (moderate from rating curve NRLDB),SAMPLE SIZE >= 2537.2 ft3/s (moderate from rating curve USGS Rating Depot),SAMPLE SIZE >= 3368.0 ft3/s (major from rating curve NRLDB),SAMPLE SIZE >= 3368.3 ft3/s (major from rating curve USGS Rating Depot)
I can provide more information about the test I performed, if desired. The code change was to the method @GeneralThresholdDefinition.getThresholds@; the modified code is below, shared for posterity because I plan to undo it. However, my only goal was to confirm that insufficiently identifying thresholds was the cause of the problem noted in #98837, and, indeed, it is. I also confirmed a run where the features were pooled appropriately identified the thresholds by name.
Thanks,
Hank
===================================================================
//Then we overwrite the original with calculated.
if ( calculatedThresholds != null )
{
for ( Entry<String, Double> threshold : calculatedThresholds.entrySet() )
{
if ( threshold.getValue() != null )
{
double value = calculatedUnitConversionOperator.applyAsDouble( threshold.getValue() );
thresholdMap.put( threshold.getKey() + " from rating curve " + getCalc_flow_values().getRating_curve().getSource(),
ThresholdOuter.of(
OneOrTwoDoubles.of( value ),
thresholdOperator,
dataType,
threshold.getKey() + " from rating curve " + getCalc_flow_values().getRating_curve().getSource(),
MeasurementUnit.of( desiredUnitMapper.getDesiredMeasurementUnitName() ) ) );
}
}
}
</code>
Author Name: Hank (Hank) Original Redmine Issue: 97209, https://vlab.noaa.gov/redmine/issues/97209 Original Date: 2021-10-06
See #97142. This can happen if a user evaluates NWM against USGS using NWS thresholds. For some NWM feature ids, thresholds from multiple NWS locations can be returned. The code needs to handle that; that is what #97142 is for. However, when seeing the thresholds in the output, I would like them qualified sufficiently to identify the WRDS location for which the thresholds were returned. It should also show the rating provider (e.g., USGS or NRLB) and rating source. There may be other important metadata to share.
This ticket applies to CSV2 and protobuf, I assume, but I'm not sure about the others. Images should not include that metadata as there is no good way to present it in a graphic, I think.
Thanks,
Hank
Related issue(s): #288 Redmine related issue(s): 98151, 99233, 99428