bug in the evaluate(..) function of the ConformalPredictiveSystem class

[Zeeshan-Khaliq]

Currently calling the evaluate() function of the ConformalPredictiveSystem class with argument metrics=["CRPS"] fails with the following error:

UnboundLocalError: local variable 'lower_percentile' referenced before assignment

Example call:

 score = model.evaluate(
                y_hat=t[predict_col].values,
                y=t[target_col].values,
                bins=bins_t,
                sigmas=sigma_t,
                y_min=0,
                metrics={"CRPS"})

Investigating the code shows there might be an issue in the implementation of evaluate() function.

tic = time.time()
if metrics is None:
    metrics = ["error","eff_mean","eff_med","CRPS","time_fit",
                "time_evaluate"]
    lower_percentile = (1-confidence)/2*100
    higher_percentile = (confidence+(1-confidence)/2)*100
    test_results = {}
if "CRPS" in metrics:
    results, cpds = self.predict(y_hat, sigmas=sigmas, bins=bins, y=y,
                                  lower_percentiles=lower_percentile,
                                  higher_percentiles=higher_percentile,
                                  y_min=y_min, y_max=y_max,
                                  return_cpds=True, cpds_by_bins=True)

In this case when a call is made with metrics={"CRPS"}, lower_percentile is never initiated so it fails.

[henrikbostrom]

Thanks for pointing this out! The bug has now been fixed in v. 0.5.1.

Best regards, Henrik