As a user, I don't want an NPE if I ask for an evaluation with sampling uncertainty

HankHerr-NOAA commented 1 month ago

The declaration is below. The exception:

2024-09-17T20:33:28.529+0000  [pool-110-thread-1] INFO EvaluationService - The evaluation failed with the following stack trace: 
wres.pipeline.InternalWresException: Could not complete project execution
    at wres.pipeline.Evaluator.evaluate(Evaluator.java:406)
    at wres.pipeline.Evaluator.evaluate(Evaluator.java:178)
    at wres.Functions.evaluate(Functions.java:135)
    at wres.server.EvaluationService.lambda$startEvaluation$2(EvaluationService.java:695)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: wres.pipeline.WresProcessingException: Encountered an error while processing evaluation 'ZOKI5kObQAQe0pY3M4_kubnl4ZQ': 
    at wres.pipeline.Evaluator.evaluate(Evaluator.java:835)
    at wres.pipeline.Evaluator.evaluate(Evaluator.java:377)
    ... 7 common frames omitted
Caused by: java.util.concurrent.CompletionException: java.lang.NullPointerException: Cannot invoke "org.apache.commons.math3.distribution.BinomialDistribution.sample()" because "sampler" is null
    at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
    at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770)
    ... 3 common frames omitted
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.commons.math3.distribution.BinomialDistribution.sample()" because "sampler" is null
    at wres.datamodel.bootstrap.StationaryBootstrapResampler.getFirstSampleFromSeriesThatIsNotFirstSeries(StationaryBootstrapResampler.java:398)
    at wres.datamodel.bootstrap.StationaryBootstrapResampler.generateResampleIndexesForForecastSeries(StationaryBootstrapResampler.java:284)
    at wres.datamodel.bootstrap.StationaryBootstrapResampler.generateResampleIndexes(StationaryBootstrapResampler.java:217)
    at wres.datamodel.bootstrap.StationaryBootstrapResampler.resample(StationaryBootstrapResampler.java:153)
    at wres.pipeline.pooling.PoolProcessor.getSamplingUncertaintyStatistics(PoolProcessor.java:597)
    at wres.pipeline.pooling.PoolProcessor.createStatistics(PoolProcessor.java:492)
    at wres.pipeline.pooling.PoolProcessor.get(PoolProcessor.java:151)
    at wres.pipeline.pooling.PoolProcessor.get(PoolProcessor.java:60)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
    ... 3 common frames omitted

I believe the cause could be that there is only a single RFC forecast available at that location which is in WGRFC and is only forecasted during events. However, I need to double check that and will the next time I'm on the NWC VPN.

I'll be back on the NWC VPN later today. When I am, I'll gather the predicted data and the thresholds.

Thanks

Hank

==========

label: 'WGRFC: Q - AHPS against USGS - One NWSLID - 01.01.2019 to Present - AHPS Thresholds'
observed:
  label: USGS NWIS Streamflow Observations
  sources:
  - interface: usgs nwis
    uri: https://nwis.waterservices.usgs.gov/nwis/iv
  variable:
    name: '00060'
  type: observations
predicted:
  label: WRDS AHPS API Streamflow Forecast
  sources:
  - interface: wrds ahps
    uri: https://[WRDS]/api/rfc_forecast/v2.0/forecast/streamflow
  variable:
    name: QR
  type: single valued forecasts
unit: ft3/s
reference_dates:
  minimum: '2024-07-19T00:00:00Z'
  maximum: '2024-08-18T00:00:00Z'
valid_dates:
  minimum: '2024-07-19T00:00:00Z'
  maximum: '2024-08-23T00:00:00Z'
features:
- observed: 08116850
  predicted: WCBT2
feature_service:
  uri: https://[WRDS]/api/location/v3.0/metadata
metrics:
- name: box plot of errors
- name: mean square error skill score
- name: volumetric efficiency
- name: mean absolute error
- name: sample size
- name: quantile quantile diagram
- name: mean absolute error skill score
- name: root mean square error
- name: threat score
- name: false alarm ratio
- name: probability of detection
- name: contingency table
- name: time to peak error
  summary_statistics:
  - mean
  - mean absolute
  - median
  - minimum
  - maximum
  - standard deviation
- name: time to peak relative error
  summary_statistics:
  - mean
  - mean absolute
  - median
  - minimum
  - maximum
  - standard deviation
threshold_sources:
- uri: https://[WRDS]/api/location/v3.0/nws_threshold
  operator: greater equal
  apply_to: predicted
  type: value
  parameter: flow
  provider: NWS-NRLDB
  rating_provider: NRLDB
  feature_name_from: predicted
sampling_uncertainty:
  sample_size: 1000
  quantiles:
  - 0.05
  - 0.95
duration_format: hours
output_formats:
- format: png
  width: 1200
  height: 1080
- format: csv2
- format: pairs
- format: svg

HankHerr-NOAA commented 1 month ago

Note that the COWRES job id for the evaluation was 8020285054067456767, and it was a WGRFC job on 9/17 (the job id will work through the front end for two weeks).

Hank

HankHerr-NOAA commented 1 month ago

The predicted data and thresholds are attached; see the links below.

If there is a configuration issue, let me know. The declaration is in the description. This is not a high priority item.

Thanks,

Hank

318_predicted.json.gz 318_threshlds.json.gz

HankHerr-NOAA commented 1 month ago

This error cropped up when attempting to add sampling uncertainty to a NWRFC 90-day RFC forecast evaluation to support VLab ticket #120579.

Hank