TheDroplets / ensverif

A toolbox of metrics to assess the performance of ensemble simulations or forecasts
GNU General Public License v3.0
2 stars 1 forks source link

Fixing the crps_hersbach_decomposition #3

Closed vinfort closed 10 months ago

vinfort commented 10 months ago

Finding the problems in crps_hersbach_decomposition, with the objective of getting the same results than the well-established CRAN verification package (https://cran.r-project.org/web/packages/verification/index.html) seemed like a daunting task. Instead, I used ChatGPT3.5 to translate the R code automagically to python. chatGPT made a single error, failing to recognize that the index of one array started at zero in python instead of starting at one in R. Apart from that correction, I made slight changes to the style to keep it in line with the rest of the ensverif package. I tested the new code on thousands of examples and got nearly identical results than with the R code (all differences were smaller than 1e-6 when evaluating streamflow forecasts against observations that are typically larger than 1 cms). I am fairly convinced that this version works, and that the version it replaced was defective. I encourage all previous users of ensverif to assess if this correction has impacts on any published results obtained with ensverif in the past. I plan to reuse the code that I am now adding in the EHPS verification package at ECCC.