As a user, if I declare an evaluation with a list of features by left and right, but not baseline, and rely on WRDS to fill in the baseline identifier, I expect it to look at both left and right identifiers

[epag]

---

Author Name: Hank (Hank) Original Redmine Issue: 111886, https://vlab.noaa.gov/redmine/issues/111886 Original Date: 2023-01-24

---

Evidence to the contrary is in #111801-18. I'll post the declaration, below.

I had left the @baseline@ attribute out of the XML @feature@ tags, so WRDS was called. However, it was called only with the @right@ side NWM feature identifiers, and the query returned 12 features despite listing 10:

Caused by: wres.io.ingesting.PreIngestException: Response from WRDS at https://nwcal-wrds.[host]/api/location/v3.0/metadata/nwm_feature_id/22337977,8110879,9436375,8403563,22340339,8110727,8507536,22340313,11688422,8110577/?identifiers=true did not include exactly 10 locations, but had 12. By declaring a feature, WRES interprets it as an intent to use that feature in the evaluation. If the corresponding feature cannot be found or has no correlation from the featureService, WRES prefers to inform you that the declared evaluation including that feature cannot be started. Options to resolve include any of: investigate the URL above in a web browser, report to the geographic feature service team the service issue or the data issue (e.g. expected correlation), omit this feature from the declaration, include a complete feature correlation declaration, or contact the WRES team for help.

This is because two of the NWM feature ids are mapped to two features, each.

Since the WRES did not look at the @left@ identifier, which would have been enough to discern the 10 I actually wanted, it errored out with the above message.

This ticket can be resolved when the WRES looks at all sides of the evaluation to uniquely identify the desired features, instead of just one side, and only errors out per the message above after all sides are considered.

Leaving this in the backlog at a high priority, since it is a bug, but not particularly urgent to fix.

Hank

[epag]

---

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2023-01-24T16:05:13Z

---

EDIT: The delcaration, below, was fixed after original posting.

The declaration:

<?xml version='1.0' encoding='UTF-8'?>
<project name="NWM Retro Sim Reproducing Jason Results">
  <inputs>
    <left label="USGS NWIS Streamflow Observations">
      <type>observations</type>
      <source interface="usgs_nwis">https://nwis.waterservices.usgs.gov/nwis/iv</source>
      <variable>00060</variable>
    </left>
    <right featureDimension="nwm_feature_id" label="MARFC RetroSim CSVs">
      <type>simulations</type>
      <source>/home/ISED/wres/nwm_2_1_retro_simulations/rfc/MARFC/5907079_nwm_2_1_retro_wres.csv.gz</source>
      <variable>streamflow</variable>
      <existingTimeScale>
        <function>mean</function>
        <period>1</period>
        <unit>seconds</unit>
      </existingTimeScale>
    </right>
    <baseline label="Persistence 1 Step USGS Streamflow">
      <type>observations</type>
      <source interface="usgs_nwis">https://nwis.waterservices.usgs.gov/nwis/iv</source>
      <variable>00060</variable>
      <persistence>1</persistence>
    </baseline>
  </inputs>
  <pair label="Retrosim Pair Reproduce Jason">
    <unit>ft3/s</unit>
    <featureService>
      <baseUrl>https://nwcal-wrds.[host]/api/location/v3.0/metadata</baseUrl>
    </featureService>
<feature left="0158175320" right="11688422"/>
<feature left="01410500" right="9436375"/>
<feature left="01521500" right="8110879"/>
<feature left="375723075344404" right="8403563"/>
<feature left="01651000" right="22337977"/>
<feature left="01653000" right="22340339"/>
<feature left="01525981" right="8110727"/>
<feature left="421512077472801" right="8110577"/>
<feature left="01652500" right="22340313"/>
<feature left="01672500" right="8507536"/>
    <dates earliest="1980-01-01T00:00:00Z" latest="2021-01-01T00:00:00Z"/>
    <desiredTimeScale lenient="none">
      <function>maximum</function>
      <period>24</period>
      <unit>hours</unit>
      <frequency>24</frequency>
    </desiredTimeScale>
  </pair>
  <metrics>
    <thresholds>
      <type>probability</type>
      <applyTo>right</applyTo>
      <commaSeparatedValues>0.5,0.55,0.6,0.65,0.7,0.75,0.8,0.85,0.9,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99,0.995,0.999</commaSeparatedValues>
      <operator>greater than</operator>
    </thresholds>
    <metric>
      <name>volumetric efficiency</name>
    </metric>
    <metric>
      <name>sample size</name>
    </metric>
    <metric>
      <name>mean absolute error skill score</name>
    </metric>
    <metric>
      <name>probability of detection</name>
    </metric>
    <metric>
      <name>false alarm ratio</name>
    </metric>
    <metric>
      <name>bias fraction</name>
    </metric>
  </metrics>
  <outputs durationFormat="hours">
    <destination type="graphic"/>
    <destination type="pairs"/>
    <destination type="numeric"/>
    <destination type="csv2"/>
  </outputs>
</project>
</code>