RuntimeError with the pp_bqn function

[tourniert]

Hello Mr Schulz, I read your article and I want to test the scripts you have developed, they work perfectly for the Histogram Estimation Network method, the Distributional Regression Network method, thank you for this work. However I get an error for the Bernstein Quantile Network method.

I run pp_bqn.R with this script (which is almost same as pp_drn.R) :

## File with exemplary usage of BQN function

# Load training and test data
load(file = paste0(getwd(), "/df_train.RData"))
load(file = paste0(getwd(), "/df_test.RData"))

# Set validation set
i_valid <- 800:984

# Import postprocessing function
source(paste0(getwd(), "/pp_bqn.R"))

# Use variables besides observations and individual ensemble members as predictors
pred_vars <- names(df_train)[!is.element(names(df_train), c("obs", paste0("ens_", 1:20)))]

# Apply QN function
pred <- bqn_pp(train = df_train, 
               X = df_test, 
               i_valid = i_valid, 
               loc_id_vec = paste0(1:10),
               pred_vars = pred_vars,
               nn_ls = list(n_sim = 3,
                            nn_verbose = TRUE))

# Take a look at some forecasts
head(pred[["f"]])

# Take a look at evaluation measures of postprocessed forecasts
summary(pred[["scores_pp"]])

# Calculate CRPSS w.r.t. ensemble forecasts (in %)
print(100*(1 - mean(pred[["scores_pp"]][["crps"]])/mean(pred[["scores_ens"]][["crps"]])))

Nevertheless I obtain the following error :

> # Apply QN function
> pred <- bqn_pp(train = df_train, 
+                X = df_test, 
+                i_valid = i_valid, 
+                loc_id_vec = paste0(1:10),
+                pred_vars = pred_vars,
+                nn_ls = list(n_sim = 3,
+                             nn_verbose = TRUE))
Epoch 1/150
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  RuntimeError: in user code:

    File "C:\Users\theo-\anaconda3\lib\site-packages\keras\engine\training.py", line 1021, in train_function  *
        return step_function(self, iterator)
    File "C:\Users\theo-\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\call.py", line 21, in python_function  *
        raise RuntimeError(res[kErrorKey])

    RuntimeError: Evaluation error: `axis` argument is 1 based, received 0.

Called from: py_call_impl(callable, dots$args, dots$keywords)
Browse[1]>

Did you have an idea to fix this problem ?

[benediktschulz]

Hello,

I was not able to recreate the error, the code you provided worked fine on my devices.

Therefore, I can only speculate where the error comes from. Maybe you are using different versions of Keras, Tensorflow or R. As far as I understand the error, there seems to be an issue in the evaluation of the loss function concerning the compatibility of the variables. The 'axis'-argument is part of the 'k_cumsum'-function within the loss-function, so maybe it does not fit to the 'y_pred'-variable anymore (in another version of keras/tensorflow or R).

I hope that helps.

[tourniert]

Thank you for your reply. Could you give me the version of R, Python, TensorFlow and Keras you are using on your devices please ?

[benediktschulz]

I am using the following versions: R (version 3.6.1), Keras (v2.4.3; R-package version 2.3.0.0), TensorFlow (v2.3.0; R-package version 2.2.0), based on Python (v3.6). The implementation of DRN makes use of the tf-probability extension (0.11.1).

[tourniert]

With the right versions of the packages, it works perfectly! Thanks very much !