search

egenn / rtemis

Advanced Machine Learning and Visualization
https://rtemis.org
GNU General Public License v3.0
138 stars 19 forks source link

shap value #63

Closed sugarballer closed 2 months ago

sugarballer commented 2 months ago

Can you add shap value function? At present, the data generated by rtemis cannot be analyzed by shapviz.

egenn commented 2 months ago

H - you can access the trained model under obj$$mod, if obj is an rtemis supervised model. You can use your preferred package/function to generate SHAP values and visualize them using that model and your data.

sugarballer commented 2 months ago

H - you can access the trained model under obj$$mod, if obj is an rtemis supervised model. You can use your preferred package/function to generate SHAP values and visualize them using that model and your data. it still doesn't work.

library(shapviz) shp <- shapviz(mod.RF$mod, X_pred = data.matrix(Sonar), X = Sonar) Error in shapviz.default(mod.RF$mod, X_pred = data.matrix(Sonar), X = Sonar) : No default method available.

library(rtemis) library(data.table) Sonar <- fread("rNPC_12.txt",data.table = F) str(Sonar) 'data.frame': 2789 obs. of 13 variables: $ gender : int 1 1 1 2 2 2 1 2 1 2 ... $ age : int 53 64 69 45 72 65 46 44 53 38 ... $ T : int 3 3 4 1 3 2 1 3 4 3 ... $ N : int 1 1 0 1 1 2 2 1 2 0 ... $ IC : int 1 1 0 0 1 1 1 0 1 1 ... $ Targeted : int 0 1 1 0 0 0 1 0 1 1 ... $ residue : int 1 0 1 0 0 0 0 0 0 0 ... $ EB_DNA_pre : num 3780 178 3670 212 110 752 427 35.5 13500 97.5 ... $ EB_DNA_preRT : int 1 0 1 1 1 1 0 1 0 0 ... $ EB_DNA_afterRT: int 1 1 0 0 1 0 0 0 0 0 ... $ CC : int 0 0 0 1 0 1 1 1 1 0 ... $ AC_real : int 0 0 0 0 0 1 0 0 0 0 ... $ Class : chr "NR" "ER" "NR" "NR" ... for(i in c(2,8)){

  • Sonar[[i]] <- as.numeric(Sonar[[i]])
  • } for(i in c(1,3,4,5,6,7,9,10,11,12)){
  • Sonar[[i]] <- as.factor(Sonar[[i]])
  • }

    Sonar$Class <- as.factor(Sonar$Class)

    Sonar$Class <- factor(Sonar$Class,levels = c("NR","ER","LR")) str(Sonar) 'data.frame': 2789 obs. of 13 variables: $ gender : Factor w/ 2 levels "1","2": 1 1 1 2 2 2 1 2 1 2 ... $ age : num 53 64 69 45 72 65 46 44 53 38 ... $ T : Factor w/ 4 levels "1","2","3","4": 3 3 4 1 3 2 1 3 4 3 ... $ N : Factor w/ 4 levels "0","1","2","3": 2 2 1 2 2 3 3 2 3 1 ... $ IC : Factor w/ 2 levels "0","1": 2 2 1 1 2 2 2 1 2 2 ... $ Targeted : Factor w/ 2 levels "0","1": 1 2 2 1 1 1 2 1 2 2 ... $ residue : Factor w/ 2 levels "0","1": 2 1 2 1 1 1 1 1 1 1 ... $ EB_DNA_pre : num 3780 178 3670 212 110 752 427 35.5 13500 97.5 ... $ EB_DNA_preRT : Factor w/ 2 levels "0","1": 2 1 2 2 2 2 1 2 1 1 ... $ EB_DNA_afterRT: Factor w/ 2 levels "0","1": 2 2 1 1 2 1 1 1 1 1 ... $ CC : Factor w/ 2 levels "0","1": 1 1 1 2 1 2 2 2 2 1 ... $ AC_real : Factor w/ 2 levels "0","1": 1 1 1 1 1 2 1 1 1 1 ... $ Class : Factor w/ 3 levels "NR","ER","LR": 1 2 1 1 1 1 1 1 1 3 ...

    Sonar$Class <- factor(Sonar$Class,levels = c("H","L"))

    Sonar <- preprocess(Sonar, impute = TRUE,scale = T) 07-09-24 16:53:25 Hello, huangzongwei [preprocess] 07-09-24 16:53:26 Imputing missing values using predictive mean matching with missRanger... [preprocess]

Missing value imputation by random forests

Variables to impute:
07-09-24 16:53:26 Scaling and Centering 2 numeric features... [preprocess] 07-09-24 16:53:26 Completed in 0.01 minutes (Real: 0.54; User: 0.02; System: 0.01) [preprocess]

check_data(Sonar) Sonar: A data.table with 2789 rows and 13 columns

Data types

07-09-24 16:53:36 Imbalanced classes: using Inverse Frequency Weighting [prepare_data]

.:Classification Input Summary Training features: 2789 x 12 Training outcome: 2789 x 1 Testing features: Not available Testing outcome: Not available

.:Parameters n.trees: 1000 mtry: NULL

07-09-24 16:53:36 Training Random Forest Classification with 1000 trees... [s_RF] ntree OOB 1 2 3 100: 12.55% 2.83%100.00% 97.73% 200: 11.72% 1.83%100.00% 99.24% 300: 11.40% 1.52%100.00% 98.48% 400: 11.19% 1.28%100.00% 98.48% 500: 11.15% 1.24%100.00% 98.48% 600: 11.15% 1.16%100.00%100.00% 700: 11.12% 1.16%100.00% 99.24% 800: 11.12% 1.16%100.00% 99.24% 900: 11.08% 1.12%100.00% 99.24% 1000: 11.08% 1.12%100.00% 99.24%

.:RF Classification Training Summary Reference Estimated NR ER LR
NR 2495 0 0 ER 7 150 0 LR 5 0 132

               Overall

Balanced Accuracy 0.9984 F1 Mean 0.9854 Accuracy 0.9957 

               NR      ER      LR      
  Sensitivity  0.9952  1.0000  1.0000
  Specificity  1.0000  0.9973  0.9981

Balanced Accuracy 0.9976 0.9987 0.9991 PPV 1.0000 0.9554 0.9635 NPV 0.9592 1.0000 1.0000 F1 0.9976 0.9772 0.9814 07-09-24 16:53:43 Completed in 0.13 minutes (Real: 7.72; User: 4.42; System: 2.20) [s_RF]

library(shapviz) shp <- shapviz(mod.RF$mod, X_pred = data.matrix(Sonar), X = Sonar) Error in shapviz.default(mod.RF$mod, X_pred = data.matrix(Sonar), X = Sonar) : No default method available.

egenn commented 2 months ago

Looks like you are providing an unsupported model. Check the shapviz documentation.

sugarballer commented 2 months ago

Looks like you are providing an unsupported model. Check the shapviz documentation.

Thanks.Looking forward to supplementing Shap's functions