search

USGS-R / river-dl

Deep learning model for predicting environmental variables on river systems
Creative Commons Zero v1.0 Universal
21 stars 15 forks source link

swapping val and tst partitions? #147

Closed janetrbarclay closed 2 years ago

janetrbarclay commented 2 years ago

In switching the default evaluation, the validation predictions are now being passed to combine_metrics as the test predictions. (see current version below). Was this intentional? Would it be better to pass in all 3 (though there was some conversation to the contrary in #141 suggesting that tst metrics not be calculated until later) or to pass the validation predictions to pred_val and let pred_tst be None (the default value). As it is now, the metrics for the validation prediction are labeled as being for the tst partition in the various metric csv files, which is confusing. https://github.com/USGS-R/river-dl/blob/c1fa37b30a291b73d4d57de7ac41dcc7ee068e23/Snakefile#L144-L159

SimonTopp commented 2 years ago

Oop, I think this was an oversight on my end. I switched it to test initially, then went switched it back after the conversation in #141. pred_tst should be pred_val I believe.