biomodhub / biomod2

BIOMOD is a computer platform for ensemble forecasting of species distributions, enabling the treatment of a range of methodological uncertainties in models and the examination of species-environment relationships.
85 stars 22 forks source link

TSS of RF and GAM return 1 #441

Closed ShreePoudel0 closed 4 months ago

ShreePoudel0 commented 6 months ago

Hello, I'm trying to do a species distribution modeling with BIOMOD2 package. I am encountering a problem regarding TSS value, more specifically the TSS value of RF and GAM are 1 or close to 1 and the TSS of ensemble model is lesser than individual model. I dont know where the problem is. My study area is 650 sq km. I have 110 presence points and am generating 300 PA (though have tried with multiple PA) myRespName<- 'Presence' myResp <- as.numeric(data[,myRespName]) myRespXY <- data[,c("X","Y")] myBiomodData <- BIOMOD_FormatingData(
resp.var = data["Presence"], resp.xy = data[, c('Y', 'X')], expl.var = myExpl, resp.name = "Presence", PA.nb.rep = 3, PA.nb.absences = 500, PA.strategy = 'random' ) myBiomodOption <- BIOMOD_ModelingOptions() myBiomodModelOut <- BIOMOD_Modeling(bm.format = myBiomodData, models = c('GLM','GBM','GAM','CTA','ANN','FDA','MARS','SRE','RF', 'MAXENT' ), bm.options = myBiomodOption, var.import = 1, CV.strategy = 'random', CV.nb.rep = 3, CV.perc = 0.7, prevalence=NULL, metric.eval = c('TSS')) myBiomodModelEval <- get_evaluations (myBiomodModelOut) myBiomodModelEval myBiomodEM <- BIOMOD_EnsembleModeling(bm.mod = myBiomodModelOut, models.chosen = 'all', em.by = "all", em.algo = c('EMmean'), metric.select = c('TSS'), metric.select.thresh = c(0.6), metric.eval = c('TSS'), var.import = 2, EMci.alpha = 0.05, EMwmean.decay = 'proportional' ) I dont know if the problem is with code, modeling or data itself.

HeleneBlt commented 6 months ago

Hi,

It seems you have some overfitting for RF and GAM.

You can try

For GAM, you can play with the smoothness of the curve. You can switch to gam with the package mgcv if it is easier for you with BIOMOD_ModelingOptions(algo = 'GAM_mgcv' )

Please let me know if you think this is not a case of overfitting or if you have any other questions.

Hélène