search

biomodhub / biomod2

BIOMOD is a computer platform for ensemble forecasting of species distributions, enabling the treatment of a range of methodological uncertainties in models and the examination of species-environment relationships.
87 stars 22 forks source link

R biomod2 error in {: task 1 failed - "<text>:1:22: unexpected input #501

Closed Qingyiyaaa closed 6 days ago

Qingyiyaaa commented 2 months ago

I am working on biomod2 but there may be something wrong with the codes. The codes are below:

DataSpecies<-read.csv("E:/otter and connectivity/Roperation/occurrence/dongbei_occurrence.csv") head(DataSpecies) myRespName<-'presence' myResp<-as.numeric(DataSpecies[,myRespName]) myRespXY<-DataSpecies[,c("X","Y")]

path<-("E:/otter and connectivity/Roperation/data(0_1)") mylist<-list.files(path,pattern='.tif$',all.files = T,full.names = T) myExpl<-terra::rast(mylist)

myBiomodData<-BIOMOD_FormatingData(resp.var = myResp, expl.var = myExpl, resp.xy = myRespXY, resp.name = myRespName, PA.nb.rep = 10, PA.nb.absences = 1000, PA.strategy = 'random', na.rm = F) myBiomodData

myBiomodModelOut<-BIOMOD_Modeling(bm.format = myBiomodData, modeling.id = 'ALLMODEL', models = c('ANN'), prevalence = 0.5, CV.strategy = 'random', CV.nb.rep = 10, CV.perc = 0.7, metric.eval = c('TSS','ROC'))

But the console showed that: error in {:task 1 failed-":1:22:unexpected input 1:presence ~1+data(0_^"

What is the problem with the codes?And how could I correct it? Thank you for your answers!

HeleneBlt commented 2 months ago

Hello there!

Thanks for reporting :blush:

So I can help you, could you share the output of show(myBiomodData) and sessionInfo() ?

Thanks a lot ! Hélène

Qingyiyaaa commented 2 months ago

Dear Hélène, I am very gald to receive your mail. The output of show(myBiomodData) and sessionInfo() are attached below. Thank you for your help! Looking forward to your reply!

Qingyi     ------------------ Original ------------------ From: @.>; Date:  Tue, Sep 10, 2024 03:29 PM To: @.>; Cc: @.>; @.>; Subject:  Re: [biomodhub/biomod2] R biomod2 error in {: task 1 failed - "<text>:1:22: unexpected input (Issue #501)

 

Hello there!

Thanks for reporting 😊

So I can help you, could you share the output of show(myBiomodData) and sessionInfo() ?

Thanks a lot ! Hélène

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

HeleneBlt commented 2 months ago

Hello again Qingyi !

Sorry but I don't have access to your attachements : you need to reply on the Github issue 😬 You can copy-paste directly on it !

Hélène

Qingyiyaaa commented 2 months ago

Dear Hélène, I am very gald to receive your mail. The output of show(myBiomodData) and sessionInfo() are listed below.

-=-=--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= BIOMOD.formated.data -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

dir.name = .

sp.name = presence

 269 presences,  0 true absences and  9979 undefined points in dataset

 20 explanatory variables

data(0_1)bioc15.tif data(0_1)bioc18.tif data(0_1)bioc2.tif data(0_1)bioc9.tif data(0_1)H_dam distance.tif Min. :0.02439 Min. :0.009799 Min. :0.02778 Min. :0.02239 Min. :0.001376
1st Qu.:0.52683 1st Qu.:0.167989 1st Qu.:0.70370 1st Qu.:0.26866 1st Qu.:0.060679
Median :0.63902 Median :0.211713 Median :0.74074 Median :0.39179 Median :0.133347
Mean :0.63760 Mean :0.218937 Mean :0.74160 Mean :0.40446 Mean :0.222907
3rd Qu.:0.75610 3rd Qu.:0.257303 3rd Qu.:0.78704 3rd Qu.:0.52612 3rd Qu.:0.299074
Max. :0.99675 Max. :0.844797 Max. :0.99074 Max. :0.97015 Max. :0.986961

data(0_1)H_population density.tif data(0_1)H_railway distance.tif data(0_1)H_residential area distance.tif Min. :0.0000000 Min. :0.00000 Min. :0.00000
1st Qu.:0.0000000 1st Qu.:0.03399 1st Qu.:0.05276
Median :0.0000145 Median :0.08416 Median :0.07682
Mean :0.0008287 Mean :0.11709 Mean :0.10538
3rd Qu.:0.0001740 3rd Qu.:0.16352 3rd Qu.:0.12663
Max. :0.2067310 Max. :0.96880 Max. :0.89006

data(0_1)H_road distance.tif data(0_1)L_aspect.tif data(0_1)L_elevation.tif data(0_1)L_lu1density.tif Min. :0.00000 Min. :0.0000 Min. :0.04704 Min. :0.00000
1st Qu.:0.02790 1st Qu.:0.2308 1st Qu.:0.12037 1st Qu.:0.03001
Median :0.06238 Median :0.4729 Median :0.19074 Median :0.20501
Mean :0.08524 Mean :0.4822 Mean :0.21593 Mean :0.29608
3rd Qu.:0.11502 3rd Qu.:0.7283 3rd Qu.:0.29185 3rd Qu.:0.51866
Max. :0.93567 Max. :0.9995 Max. :0.74926 Max. :0.97981
NA's :3
data(0_1)L_lu2density.tif data(0_1)L_slope.tif data(0_1)L_VRM.tif data(0_1)W_river1distance.tif data(0_1)W_river2distance.tif Min. :0.00000 Min. :0.00000 Min. :0.000000 Min. :0.0000 Min. :0.00000
1st Qu.:0.06519 1st Qu.:0.01348 1st Qu.:0.000347 1st Qu.:0.1382 1st Qu.:0.08074
Median :0.38151 Median :0.05005 Median :0.004584 Median :0.2940 Median :0.18902
Mean :0.41621 Mean :0.07923 Mean :0.017065 Mean :0.3237 Mean :0.23475
3rd Qu.:0.75950 3rd Qu.:0.11519 3rd Qu.:0.020972 3rd Qu.:0.4826 3rd Qu.:0.34685
Max. :0.99696 Max. :0.67074 Max. :0.605808 Max. :0.9781 Max. :0.99222
NA's :3 NA's :5
data(0_1)W_river3distance.tif data(0_1)W_river4distance.tif data(0_1)W_water area distance.tif Min. :0.00000 Min. :0.00000 Min. :0.00000
1st Qu.:0.04687 1st Qu.:0.06485 1st Qu.:0.01608
Median :0.09709 Median :0.12697 Median :0.03320
Mean :0.11971 Mean :0.16620 Mean :0.10067
3rd Qu.:0.16985 3rd Qu.:0.22568 3rd Qu.:0.07751
Max. :0.93551 Max. :0.94597 Max. :0.99589

10 Pseudo Absences dataset available ( PA1, PA2, PA3, PA4, PA5, PA6, PA7, PA8, PA9, PA10 ) with
1000 (PA1, PA2, PA3, PA4, PA5, PA6, PA7, PA8, PA9, PA10) pseudo absences

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

sessionInfo() R version 4.4.1 (2024-06-14 ucrt) Platform: x86_64-w64-mingw32/x64 Running under: Windows 11 x64 (build 22631)

Matrix products: default

locale: [1] LC_COLLATE=Chinese (Simplified)_China.utf8 LC_CTYPE=Chinese (Simplified)_China.utf8
[3] LC_MONETARY=Chinese (Simplified)_China.utf8 LC_NUMERIC=C
[5] LC_TIME=Chinese (Simplified)_China.utf8

time zone: Asia/Shanghai tzcode source: internal

attached base packages: [1] splines stats graphics grDevices utils datasets methods base

other attached packages: [1] raster_3.6-26 sp_2.1-4 terra_1.7-78 xgboost_1.7.8.1 randomForest_4.7-1.1 [6] maxnet_0.1.4 earth_5.3.3 plotmo_3.6.4 plotrix_3.8-4 Formula_1.2-5
[11] gbm_2.2.2 mgcv_1.9-1 nlme_3.1-164 gam_1.22-4 foreach_1.5.2
[16] mda_0.5-4 class_7.3-22 rpart_4.1.23 nnet_7.3-19 biomod2_4.2-5-2

loaded via a namespace (and not attached): [1] gtable_0.3.5 ggplot2_3.5.1 htmlwidgets_1.6.4 devtools_2.4.5 remotes_2.5.0
[6] lattice_0.22-6 vctrs_0.6.5 tools_4.4.1 generics_0.1.3 tibble_3.2.1
[11] PresenceAbsence_1.1.11 fansi_1.0.6 pkgconfig_2.0.3 Matrix_1.7-0 data.table_1.16.0
[16] lifecycle_1.0.4 compiler_4.4.1 stringr_1.5.1 munsell_0.5.1 codetools_0.2-20
[21] httpuv_1.6.15 htmltools_0.5.8.1 usethis_3.0.0 later_1.3.2 pillar_1.9.0
[26] urlchecker_1.0.1 ellipsis_0.3.2 cachem_1.1.0 sessioninfo_1.2.2 iterators_1.0.14
[31] abind_1.4-5 mime_0.12 tidyselect_1.2.1 digest_0.6.37 stringi_1.8.4
[36] dplyr_1.1.4 reshape2_1.4.4 purrr_1.0.2 fastmap_1.2.0 grid_4.4.1
[41] colorspace_2.1-1 cli_3.6.3 magrittr_2.0.3 survival_3.6-4 pkgbuild_1.4.4
[46] utf8_1.2.4 scales_1.3.0 promises_1.3.0 memoise_2.0.1 shiny_1.9.1
[51] miniUI_0.1.1.1 profvis_0.3.8 rlang_1.1.4 Rcpp_1.0.13 xtable_1.8-4
[56] glue_1.7.0 pROC_1.18.5 pkgload_1.4.0 jsonlite_1.8.8 rstudioapi_0.16.0
[61] reshape_0.8.9 R6_2.5.1 plyr_1.8.9 fs_1.6.4

Thank you for your help! Looking forward to your reply!

HeleneBlt commented 2 months ago

I see where the problem come from ! You will need to change the names of your variables. If you keep data(0_1)bioc15.tif for example, it will lead to some error in the formula for the models. It will try to interpret the parentheses and the point. I will advise changing it to bioc15.

Hope it helps ! Hélène

Qingyiyaaa commented 2 months ago

Dear Hélène, It does work! But I met a new problem. The error is listed below:

-=-=-=--=-=-=- presence_allData_allRun_ANN

weights: 45

initial value 243.981236 iter 10 value 98.335033 iter 20 value 86.144772 iter 30 value 84.471712 iter 40 value 83.675508 iter 50 value 82.421309 iter 60 value 82.094954 iter 70 value 81.781016 iter 80 value 81.470009 iter 90 value 81.270757 iter 100 value 81.083984 final value 81.083984 stopped after 100 iterations

Model scaling...
Evaluating Model stuff...
Note : some NA occurs in predictionsNon finite obs or fit available => model evaluation skipped !Non finite obs or fit available => model evaluation skipped !error in {: task 11 failed - "$ operator is invalid for atomic vectors"

Thank you for your help! Looking forward to your reply!

HeleneBlt commented 2 months ago

Hello Qingyi !

I think the problem comes from the NAs in your myExpl. Some algorithms handle it well, but that's not the case with ANN. You will need to set na.rm = T when you format your data. 

myBiomodData<-BIOMOD_FormatingData(resp.var = myResp,
                                   expl.var = myExpl,
                                   resp.xy = myRespXY,
                                   resp.name = myRespName,
                                   PA.nb.rep = 10,
                                   PA.nb.absences = 1000,
                                   PA.strategy = 'random',
                                   na.rm = T)

Hope it helps ☺️ Hélène

Qingyiyaaa commented 1 month ago

Dear Hélène, That does help! Thank you very much! And is it possible to calculate AUC value with Biomod2? As an evaluation value, you can get ROC: relative operating characteristic value. I don't quite understand how to calculate AUC. Maybe I just don't understand the connection between ROC, sensitivity and specificity. Thank you for your time! Looking forward to your reply!

HeleneBlt commented 1 month ago

Hello !

We call this metric ROC to explain clearly that we create the ROC curve, but the value at the end is AUC. The threshold, sensitivity and specificity that biomod2 returns with the ROC/AUC value are the ones associated with the best point : image

Here biomod2 will return

Hope it is clearer this way 💪 Hélène

(Figure from Pat-Horenczyk, Ruth & Kenan, Avraham & Achituv, Michal & Bachar, Eytan. (2013). Protective Factors Based Model for Screening for Posttraumatic Distress in Adolescents. Child & Youth Care Forum. 43. 10.1007/s10566-013-9241-y. )

Qingyiyaaa commented 1 month ago

Dear Hélène, Thank you for your time! So is the "calibration" the value of AUC?

                                    full.name      PA    run algo metric.eval cutoff sensitivity specificity calibration validation

1 presence_PA1_RUN1_CTA PA1 RUN1 CTA ROC 613.0 79.255 91.429 0.905 0.849 2 presence_PA1_RUN2_CTA PA1 RUN2 CTA ROC 548.0 86.702 75.714 0.835 0.809 3 presence_PA1_RUN3_CTA PA1 RUN3 CTA ROC 529.0 87.234 88.571 0.911 0.829 4 presence_PA1_RUN4_CTA PA1 RUN4 CTA ROC 451.0 87.234 74.286 0.808 0.757 5 presence_PA1_RUN5_CTA PA1 RUN5 CTA ROC 476.5 86.702 83.333 0.871 0.863 6 presence_PA1_RUN6_CTA PA1 RUN6 CTA ROC 569.5 84.043 76.190 0.817 0.791 7 presence_PA1_RUN7_CTA PA1 RUN7 CTA ROC 414.5 88.298 64.286 0.763 0.766 8 presence_PA1_RUN8_CTA PA1 RUN8 CTA ROC 559.0 87.234 89.524 0.928 0.863 9 presence_PA1_RUN9_CTA PA1 RUN9 CTA ROC 565.5 87.766 76.190 0.848 0.825 10 presence_PA1_RUN10_CTA PA1 RUN10 CTA ROC 430.5 85.106 80.000 0.840 0.804

Looking forward to your reply!

HeleneBlt commented 1 month ago

Yes exactly! The column calibration is the AUC value for the calibration part of your cross-validation and validation for the validation part of your CV.

Hélène

Qingyiyaaa commented 1 month ago

Dear Hélène, Yes!It does work!But I have a new question.Why does the value of TSS and AUC of RF close to 1 or just 1? I tried to change the parameters, such as "nodesize", "PA.nb.absences", "CV.perc", but little work. So how could I do to address the problem?

get_evaluations(myBiomodModelOut) full.name PA run algo metric.eval cutoff sensitivity specificity calibration validation evaluation 1 presence_PA1_RUN1_RF PA1 RUN1 RF TSS 525.0 100 100 1 0.683 NA 2 presence_PA1_RUN2_RF PA1 RUN2 RF TSS 504.0 100 100 1 0.609 NA 3 presence_PA1_RUN3_RF PA1 RUN3 RF TSS 523.0 100 100 1 0.720 NA 4 presence_PA1_RUN4_RF PA1 RUN4 RF TSS 521.0 100 100 1 0.701 NA 5 presence_PA1_RUN5_RF PA1 RUN5 RF TSS 543.0 100 100 1 0.719 NA 6 presence_PA1_RUN6_RF PA1 RUN6 RF TSS 512.0 100 100 1 0.636 NA 7 presence_PA1_RUN7_RF PA1 RUN7 RF TSS 510.0 100 100 1 0.617 NA 8 presence_PA1_RUN8_RF PA1 RUN8 RF TSS 517.0 100 100 1 0.664 NA 9 presence_PA1_RUN9_RF PA1 RUN9 RF TSS 532.0 100 100 1 0.617 NA 10 presence_PA1_RUN10_RF PA1 RUN10 RF TSS 546.0 100 100 1 0.644 NA 11 presence_PA1_allRun_RF PA1 allRun RF TSS 512.0 100 100 1 NA NA 12 presence_PA2_RUN1_RF PA2 RUN1 RF TSS 520.0 100 100 1 0.655 NA 13 presence_PA2_RUN2_RF PA2 RUN2 RF TSS 528.0 100 100 1 0.627 NA 14 presence_PA2_RUN3_RF PA2 RUN3 RF TSS 533.0 100 100 1 0.580 NA 15 presence_PA2_RUN4_RF PA2 RUN4 RF TSS 521.0 100 100 1 0.609 NA 16 presence_PA2_RUN5_RF PA2 RUN5 RF TSS 522.0 100 100 1 0.672 NA 17 presence_PA2_RUN6_RF PA2 RUN6 RF TSS 544.0 100 100 1 0.571 NA 18 presence_PA2_RUN7_RF PA2 RUN7 RF TSS 535.0 100 100 1 0.570 NA 19 presence_PA2_RUN8_RF PA2 RUN8 RF TSS 527.0 100 100 1 0.682 NA 20 presence_PA2_RUN9_RF PA2 RUN9 RF TSS 515.0 100 100 1 0.683 NA 21 presence_PA2_RUN10_RF PA2 RUN10 RF TSS 512.0 100 100 1 0.618 NA 22 presence_PA2_allRun_RF PA2 allRun RF TSS 524.0 100 100 1 NA NA 23 presence_PA3_RUN1_RF PA3 RUN1 RF TSS 522.0 100 100 1 0.589 NA 24 presence_PA3_RUN2_RF PA3 RUN2 RF TSS 515.0 100 100 1 0.673 NA 25 presence_PA3_RUN3_RF PA3 RUN3 RF TSS 522.0 100 100 1 0.627 NA 26 presence_PA3_RUN4_RF PA3 RUN4 RF TSS 492.0 100 100 1 0.758 NA 27 presence_PA3_RUN5_RF PA3 RUN5 RF TSS 542.0 100 100 1 0.495 NA 28 presence_PA3_RUN6_RF PA3 RUN6 RF TSS 526.0 100 100 1 0.682 NA 29 presence_PA3_RUN7_RF PA3 RUN7 RF TSS 523.0 100 100 1 0.598 NA 30 presence_PA3_RUN8_RF PA3 RUN8 RF TSS 543.0 100 100 1 0.682 NA 31 presence_PA3_RUN9_RF PA3 RUN9 RF TSS 509.0 100 100 1 0.637 NA 32 presence_PA3_RUN10_RF PA3 RUN10 RF TSS 519.0 100 100 1 0.693 NA 33 presence_PA3_allRun_RF PA3 allRun RF TSS 529.0 100 100 1 NA NA 34 presence_PA4_RUN1_RF PA4 RUN1 RF TSS 512.0 100 100 1 0.507 NA 35 presence_PA4_RUN2_RF PA4 RUN2 RF TSS 500.0 100 100 1 0.580 NA 36 presence_PA4_RUN3_RF PA4 RUN3 RF TSS 512.0 100 100 1 0.617 NA 37 presence_PA4_RUN4_RF PA4 RUN4 RF TSS 509.0 100 100 1 0.607 NA 38 presence_PA4_RUN5_RF PA4 RUN5 RF TSS 519.0 100 100 1 0.600 NA 39 presence_PA4_RUN6_RF PA4 RUN6 RF TSS 506.0 100 100 1 0.544 NA 40 presence_PA4_RUN7_RF PA4 RUN7 RF TSS 494.0 100 100 1 0.684 NA 41 presence_PA4_RUN8_RF PA4 RUN8 RF TSS 505.0 100 100 1 0.580 NA 42 presence_PA4_RUN9_RF PA4 RUN9 RF TSS 495.0 100 100 1 0.628 NA 43 presence_PA4_RUN10_RF PA4 RUN10 RF TSS 510.0 100 100 1 0.553 NA 44 presence_PA4_allRun_RF PA4 allRun RF TSS 513.0 100 100 1 NA NA 45 presence_PA5_RUN1_RF PA5 RUN1 RF TSS 527.0 100 100 1 0.692 NA 46 presence_PA5_RUN2_RF PA5 RUN2 RF TSS 537.0 100 100 1 0.645 NA 47 presence_PA5_RUN3_RF PA5 RUN3 RF TSS 533.0 100 100 1 0.654 NA 48 presence_PA5_RUN4_RF PA5 RUN4 RF TSS 527.0 100 100 1 0.617 NA 49 presence_PA5_RUN5_RF PA5 RUN5 RF TSS 502.0 100 100 1 0.673 NA 50 presence_PA5_RUN6_RF PA5 RUN6 RF TSS 545.0 100 100 1 0.598 NA 51 presence_PA5_RUN7_RF PA5 RUN7 RF TSS 530.0 100 100 1 0.598 NA 52 presence_PA5_RUN8_RF PA5 RUN8 RF TSS 538.0 100 100 1 0.664 NA 53 presence_PA5_RUN9_RF PA5 RUN9 RF TSS 505.5 100 100 1 0.572 NA 54 presence_PA5_RUN10_RF PA5 RUN10 RF TSS 516.0 100 100 1 0.534 NA 55 presence_PA5_allRun_RF PA5 allRun RF TSS 534.5 100 100 1 NA NA 56 presence_PA6_RUN1_RF PA6 RUN1 RF TSS 507.5 100 100 1 0.617 NA 57 presence_PA6_RUN2_RF PA6 RUN2 RF TSS 515.0 100 100 1 0.684 NA 58 presence_PA6_RUN3_RF PA6 RUN3 RF TSS 553.0 100 100 1 0.654 NA 59 presence_PA6_RUN4_RF PA6 RUN4 RF TSS 493.0 100 100 1 0.674 NA 60 presence_PA6_RUN5_RF PA6 RUN5 RF TSS 538.0 100 100 1 0.608 NA 61 presence_PA6_RUN6_RF PA6 RUN6 RF TSS 513.0 100 100 1 0.646 NA 62 presence_PA6_RUN7_RF PA6 RUN7 RF TSS 503.0 100 100 1 0.590 NA 63 presence_PA6_RUN8_RF PA6 RUN8 RF TSS 509.0 100 100 1 0.562 NA 64 presence_PA6_RUN9_RF PA6 RUN9 RF TSS 555.0 100 100 1 0.673 NA 65 presence_PA6_RUN10_RF PA6 RUN10 RF TSS 534.0 100 100 1 0.682 NA 66 presence_PA6_allRun_RF PA6 allRun RF TSS 535.0 100 100 1 NA NA 67 presence_PA7_RUN1_RF PA7 RUN1 RF TSS 524.0 100 100 1 0.644 NA 68 presence_PA7_RUN2_RF PA7 RUN2 RF TSS 545.5 100 100 1 0.646 NA 69 presence_PA7_RUN3_RF PA7 RUN3 RF TSS 502.0 100 100 1 0.628 NA 70 presence_PA7_RUN4_RF PA7 RUN4 RF TSS 535.0 100 100 1 0.523 NA 71 presence_PA7_RUN5_RF PA7 RUN5 RF TSS 523.5 100 100 1 0.645 NA 72 presence_PA7_RUN6_RF PA7 RUN6 RF TSS 520.5 100 100 1 0.617 NA 73 presence_PA7_RUN7_RF PA7 RUN7 RF TSS 508.0 100 100 1 0.692 NA 74 presence_PA7_RUN8_RF PA7 RUN8 RF TSS 509.0 100 100 1 0.608 NA 75 presence_PA7_RUN9_RF PA7 RUN9 RF TSS 512.5 100 100 1 0.665 NA 76 presence_PA7_RUN10_RF PA7 RUN10 RF TSS 498.0 100 100 1 0.572 NA 77 presence_PA7_allRun_RF PA7 allRun RF TSS 518.0 100 100 1 NA NA 78 presence_PA8_RUN1_RF PA8 RUN1 RF TSS 518.0 100 100 1 0.608 NA 79 presence_PA8_RUN2_RF PA8 RUN2 RF TSS 527.5 100 100 1 0.644 NA 80 presence_PA8_RUN3_RF PA8 RUN3 RF TSS 503.0 100 100 1 0.533 NA 81 presence_PA8_RUN4_RF PA8 RUN4 RF TSS 529.0 100 100 1 0.627 NA 82 presence_PA8_RUN5_RF PA8 RUN5 RF TSS 525.0 100 100 1 0.487 NA 83 presence_PA8_RUN6_RF PA8 RUN6 RF TSS 491.0 100 100 1 0.580 NA 84 presence_PA8_RUN7_RF PA8 RUN7 RF TSS 512.0 100 100 1 0.561 NA 85 presence_PA8_RUN8_RF PA8 RUN8 RF TSS 526.0 100 100 1 0.514 NA 86 presence_PA8_RUN9_RF PA8 RUN9 RF TSS 520.0 100 100 1 0.636 NA 87 presence_PA8_RUN10_RF PA8 RUN10 RF TSS 535.0 100 100 1 0.616 NA 88 presence_PA8_allRun_RF PA8 allRun RF TSS 530.0 100 100 1 NA NA 89 presence_PA9_RUN1_RF PA9 RUN1 RF TSS 532.0 100 100 1 0.636 NA 90 presence_PA9_RUN2_RF PA9 RUN2 RF TSS 508.0 100 100 1 0.636 NA 91 presence_PA9_RUN3_RF PA9 RUN3 RF TSS 546.0 100 100 1 0.608 NA 92 presence_PA9_RUN4_RF PA9 RUN4 RF TSS 522.0 100 100 1 0.627 NA 93 presence_PA9_RUN5_RF PA9 RUN5 RF TSS 519.0 100 100 1 0.599 NA 94 presence_PA9_RUN6_RF PA9 RUN6 RF TSS 534.0 100 100 1 0.598 NA 95 presence_PA9_RUN7_RF PA9 RUN7 RF TSS 527.0 100 100 1 0.683 NA 96 presence_PA9_RUN8_RF PA9 RUN8 RF TSS 521.0 100 100 1 0.590 NA 97 presence_PA9_RUN9_RF PA9 RUN9 RF TSS 520.5 100 100 1 0.636 NA 98 presence_PA9_RUN10_RF PA9 RUN10 RF TSS 528.5 100 100 1 0.673 NA 99 presence_PA9_allRun_RF PA9 allRun RF TSS 510.0 100 100 1 NA NA 100 presence_PA10_RUN1_RF PA10 RUN1 RF TSS 519.0 100 100 1 0.579 NA 101 presence_PA10_RUN2_RF PA10 RUN2 RF TSS 482.5 100 100 1 0.570 NA 102 presence_PA10_RUN3_RF PA10 RUN3 RF TSS 513.0 100 100 1 0.477 NA 103 presence_PA10_RUN4_RF PA10 RUN4 RF TSS 529.0 100 100 1 0.580 NA 104 presence_PA10_RUN5_RF PA10 RUN5 RF TSS 501.0 100 100 1 0.571 NA 105 presence_PA10_RUN6_RF PA10 RUN6 RF TSS 528.0 100 100 1 0.598 NA 106 presence_PA10_RUN7_RF PA10 RUN7 RF TSS 508.0 100 100 1 0.497 NA 107 presence_PA10_RUN8_RF PA10 RUN8 RF TSS 493.0 100 100 1 0.626 NA 108 presence_PA10_RUN9_RF PA10 RUN9 RF TSS 503.0 100 100 1 0.533 NA 109 presence_PA10_RUN10_RF PA10 RUN10 RF TSS 520.0 100 100 1 0.534 NA 110 presence_PA10_allRun_RF PA10 allRun RF TSS 526.0 100 100 1 NA NA 111 presence_allData_allRun_RF allData allRun RF TSS 436.0 100 100 1 NA NA

Thank for your time!

HeleneBlt commented 1 month ago

Hi Qingyi !

You have indeed some overfitting here. (Note that it is not very strong as your validation score is acceptable.) To reduce the over-fitting, you can have a look to the issue #460 for example. There are several issues with different solutions to avoid overfitting with RF. Usually by changing nodesize as you already do but also maxnodes. In the GitHub version (you can download it with devtools::[install_github](https://remotes.r-lib.org/reference/install_github.html)("biomodhub/biomod2", dependencies = TRUE), you have access to RFd: it is a RandomForest with a down-sampling method. It can be a good solution to correct overfitting.

Hope it helps, Hélène

Qingyiyaaa commented 1 month ago

Dear Hélène, Thank you for your kind reply! Imet a new problem. When I plot the ensemble model, the warning comes out.

The warning is: as.double(y): cannot coerce type 'S4' to vector of type 'double'

And the ensemble model codes are below: myBiomodModelOut<-BIOMOD_Modeling(bm.format = myBiomodData, modeling.id = 'AllModels', models = c('GBM','GLM','MARS','MAXNET'), prevalence = 0.5, CV.strategy = 'random', CV.nb.rep = 30, CV.perc = 0.8, metric.eval = c('TSS','ROC'))

myBiomodEM <- BIOMOD_EnsembleModeling(bm.mod = myBiomodModelOut, models.chosen = 'all', em.by = 'all', em.algo = c('EMwmean'), metric.select = c('TSS'), metric.select.thresh = c(0.7), metric.eval = c('TSS', 'ROC'), var.import = 3, seed.val = 42) myBiomodEM get_evaluations(myBiomodEM) get_variables_importance(myBiomodEM)

myBiomodEMProj <- BIOMOD_EnsembleForecasting(bm.em = myBiomodEM, proj.name = 'CurrentEM', new.env = myExpl, models.chosen = 'all', metric.binary = 'all', metric.filter = 'all') myBiomodEMProj plot(myBiomodEM)

Looking forward to your reply!

Qingyiyaaa commented 1 month ago

And how to have a single Ensemble Model, filtered jointly by AUC and TSS?

Thank you for your time again!

HeleneBlt commented 1 month ago

Hello !

Your object myBiomodEM is a model object and there is no plot method associated with this object. It will be more useful to plot the projections: myBiomodEmProj. :world_map:

It is not possible to filter with two metrics at the same time, sorry! However, you can have a look at the cutoff column in get_evaluations(myBiomodEM) to see if they differ much. 👀

Don't hesitate if it's not clear. Hélène