plotD3 - support for multiple models

[agosiewska]

something is wrong with displaying D3 plots

library("DALEX2")
library("breakDown2")
library("randomForest")
set.seed(1313)
model <- randomForest(status ~ . , data = HR)
new_observation <- HR_test[1,]

explainer_rf <- explain(model,
                        data = HR[1:1000,1:5],
                        y = HR$status[1:1000])

bd_rf <- local_attributions(explainer_rf,
                            new_observation)
plotD3(bd_rf)

session info

R version 3.5.2 (2018-12-20)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=Polish_Poland.1250  LC_CTYPE=Polish_Poland.1250   
[3] LC_MONETARY=Polish_Poland.1250 LC_NUMERIC=C                  
[5] LC_TIME=Polish_Poland.1250    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] breakDown2_0.9.1    reticulate_1.10     randomForest_4.6-14 iBreakDown_0.9.3   
[5] DALEX2_0.9         

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.0       rstudioapi_0.9.0 magrittr_1.5     tidyselect_0.2.5
 [5] munsell_0.5.0    lattice_0.20-38  colorspace_1.4-0 R6_2.3.0        
 [9] rlang_0.3.1      plyr_1.8.4       dplyr_0.8.0.1    tools_3.5.2     
[13] grid_3.5.2       gtable_0.2.0     htmltools_0.3.6  digest_0.6.18   
[17] yaml_2.2.0       lazyeval_0.2.1   assertthat_0.2.0 tibble_2.0.1    
[21] crayon_1.3.4     Matrix_1.2-15    purrr_0.2.5      ggplot2_3.1.0   
[25] htmlwidgets_1.3  glue_1.3.0       compiler_3.5.2   pillar_1.3.1    
[29] r2d3_0.2.3       scales_1.0.0     jsonlite_1.6     pkgconfig_2.0.2 

[pbiecek]

Thanks, current plotD3 does not support multiple models (here models for 3 classes) I hope that Hubert will fix it some day.

For now please use it for continuous variable or binary classification only

library("DALEX")
library("iBreakDown")
library("randomForest")
set.seed(1313)
model <- randomForest(status == "fired" ~ . , data = HR)
new_observation <- HR_test[1,]

explainer_rf <- explain(model,
                        data = HR[1:1000,1:5],
                        y = HR$status[1:1000] == "fired")

bd_rf <- local_attributions(explainer_rf,
                            new_observation)
plotD3(bd_rf)

[agosiewska]

Thanks, but it seems to me, that it works only for regression. You obtain regression random forest in the example above.

iBreakDown doesn't work for binary classification.

library("DALEX")
library("iBreakDown")
library("randomForest")
set.seed(1313)
# I've added factor here in random forest
model <- randomForest(factor(status == "fired") ~ . , data = HR) 
new_observation <- HR_test[1,]

explainer_rf <- explain(model,
                        data = HR[1:1000,1:5],
                        y = HR$status[1:1000] == "fired")

bd_rf <- local_attributions(explainer_rf,
                            new_observation)
plotD3(bd_rf)

[pbiecek]

Would you please try DALEX in the latest version from GitHub (DALEX_0.2.7) ?

devtools::install_github("pbiecek/DALEX")

For binary classification, by default it takes scores only for the second class (I fixed it two days ago)

[agosiewska]

Still the same. I've updated DALEX and iBreakdown

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=Polish_Poland.1250  LC_CTYPE=Polish_Poland.1250   
[3] LC_MONETARY=Polish_Poland.1250 LC_NUMERIC=C                  
[5] LC_TIME=Polish_Poland.1250    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] randomForest_4.6-14 iBreakDown_0.9.3    DALEX_0.2.7        

loaded via a namespace (and not attached):
 [1] jsonlite_1.6       splines_3.5.2      gtools_3.8.1       shiny_1.2.0       
 [5] assertthat_0.2.0   expm_0.999-3       sp_1.3-1           highr_0.7         
 [9] pdp_0.7.0          yaml_2.2.0         LearnBayes_2.15.1  r2d3_0.2.3        
[13] pillar_1.3.1       lattice_0.20-38    glue_1.3.0         digest_0.6.18     
[17] promises_1.0.1     colorspace_1.4-0   htmltools_0.3.6    httpuv_1.4.5.1    
[21] Matrix_1.2-15      plyr_1.8.4         klaR_0.6-14        pkgconfig_2.0.2   
[25] breakDown_0.1.6    questionr_0.7.0    gmodels_2.18.1     purrr_0.3.0       
[29] xtable_1.8-3       mvtnorm_1.0-8      scales_1.0.0       gdata_2.18.0      
[33] later_0.8.0        tibble_2.0.1       proxy_0.4-22       combinat_0.0-8    
[37] ggplot2_3.1.0      ggpubr_0.2         ALEPlot_1.1        agricolae_1.3-0   
[41] lazyeval_0.2.1     survival_2.43-3    magrittr_1.5       crayon_1.3.4      
[45] mime_0.6           deldir_0.1-16      nlme_3.1-137       MASS_7.3-51.1     
[49] class_7.3-14       tools_3.5.2        stringr_1.4.0      munsell_0.5.0     
[53] cluster_2.0.7-1    compiler_3.5.2     e1071_1.7-0.1      rlang_0.3.1       
[57] classInt_0.3-1     units_0.6-2        grid_3.5.2         rstudioapi_0.9.0  
[61] htmlwidgets_1.3    miniUI_0.1.1.1     boot_1.3-20        gtable_0.2.0      
[65] DBI_1.0.0          reshape2_1.4.3     AlgDesign_1.1-7.3  R6_2.4.0          
[69] yaImpute_1.0-31    gridExtra_2.3      knitr_1.21         dplyr_0.8.0.1     
[73] factorMerger_0.3.6 spdep_1.0-2        stringi_1.3.1      Rcpp_1.0.0        
[77] sf_0.7-2           spData_0.3.0       tidyselect_0.2.5   xfun_0.5          
[81] coda_0.19-2     

[agosiewska]

I've updated DALEX again and now it works :) It would be cool to have support for multiple models, so I'm leaving this issue open. Hope Hubert will do it :)

[hbaniecki]

Multiple models in both plotD3 should now supported :)