hmsc-r / hmsc-hpc

13 stars 4 forks source link

Cross-validation issue with HPC : constant AUC #17

Closed Loic-sanchez closed 1 month ago

Loic-sanchez commented 1 month ago

Hello, I have tried to run a cross-validation on HMSC-HPC (using scripts provided on https://github.com/Lord-Rivers/HMSCPipeLine), everything ran smoothly and the object I obtain in the end contains multiple objects : a wAIC, MF, and MFCV.

The model is a probit regression on 139 species, 366 sites, with a spatial random term (full method), running on only 2 chains, 200 samples, 200 thinning. When I run the full model with HPC and a prediction on the full model I obtain a range of AUC values going from 0.75 to 0.99. However when I run the cross-validation, the MF and MFCV objects are exactly the same:

Any idea as to why this happens ? Thanks

Loic-sanchez commented 1 month ago

Ran it on a remote server and the issue disappeared.