Monitoring R2 at train time

[aga-relation]

Hi!

In the paper you mention that "the stopping criterion monitored was the ‘r-squared’ value".

However, when I run the training notebook, I get the following warning: "Early stopping conditioned on metric val_r_square which is not available".

After checking the documentation, it seems that the R Squared is available in Tensorflow Addons, however tfa not compatible with tf1.14? Could you please let me know what I am missing?

Thank you very much! :)

[1edv]

Hi,

I do remember the 'r-squared' computation being quite finicky:

• The function you likely want to use can be accessed on line 1500, in the rr_aux.py file. Note that K refers to the variable imported in line 36 (from tensorflow.keras import backend as K). You should be able to adapt this function to compute R-squared for any version of tensorflow.

• If you have further issues with tensorflow, a more straightforward (but maybe also more 'hacky') way of doing this would be to simply compute r-squared using the sklearn implementation that you can access here, for each model epoch. You could simply run training for a few epochs, save the model for each epoch, compute r-squared (or any other metric of your choice) using the sklearn implementation at the end of training for each epoch on the validation data, and pick your preferred model depending on your criterion.

Good luck!

[aga-relation]

Thank you very much for your reply! I am plotting R2 at train time and find that it is very stochastic - is that the expected behaviour?

[1edv]

If you are plotting it out for each batch, the stochasticity is to be expected.

If you are running it at the end of each epoch on just the validation set, it should stabilize after a couple of epochs.