iiasa / ibis.iSDM

Modelling framework for creating Integrated SDMS
https://iiasa.github.io/ibis.iSDM/
Creative Commons Attribution 4.0 International
20 stars 2 forks source link

Implement spatial block-validation in `ibis.iSDM` ? #89

Open Martin-Jung opened 8 months ago

Martin-Jung commented 8 months ago

(Spatial) Block validation has so far not been added to the package given the complexities of assigning blocks to single or multiple datasets that might be specified in the model. Thus in most projects we usually implement the cross-validation externally, e.g. providing subsets of training data to individual ibis fits and then validate them externally. I still think outsourcing validation to the user makes the most sense.

However...., given that increasingly we have a range of projects that need to rely on this, we could brainstorm on how to best support this functionality within ibis. I judge this as a relatively big overhaul if implemented well.

So possible implementation steps:

Thoughts?

Martin-Jung commented 4 months ago

This has been dormant since a while and as we implement this already in a range of processing pipelines, I am still in favour of not adding it directly I think. Instead provide an extra vignette highlighting (spatial) cross-validation approaches might be an idea. The vignette could simply import and use the spatialsample package