search

gboris / blkbox

Data exploration with multiple machine learning algorithms
14 stars 4 forks source link

bartmachine gives varying metric with the same seed (-> hand down seed in modules) #36

Closed gboris closed 8 years ago

gboris commented 8 years ago

for ERR,ACC,MCC,F1 rest is constant

on gamma results=blkboxNCV(data=data_small, label=labels, seed= 1 )


                   ERR     AUROC       ACC       MCC       F-1
randomforest 0.2903226 0.7840909 0.7096774 0.3425801 0.7857143
kknn         0.2419355 0.7914773 0.7580645 0.4575930 0.8192771
bartmachine  0.2419355 0.7943182 0.7580645 0.4575930 0.8192771
party        0.2741935 0.7647727 0.7258065 0.3615022 0.8131868
GLM          0.1935484 0.8079545 0.8064516 0.5871472 0.8461538
PamR         0.2096774 0.8034091 0.7903226 0.5307135 0.8433735
nnet         0.2258065 0.8096591 0.7741935 0.5179397 0.8205128
SVM          0.2580645 0.8329545 0.7419355 0.4363636 0.8000000

on medusa: blackbox_output_small = blkboxNCV(data = dataset_small, labels = labels, seed =1 )


                   ERR     AUROC       ACC       MCC       F-1
randomforest 0.2903226 0.7840909 0.7096774 0.3425801 0.7857143
kknn         0.2419355 0.7914773 0.7580645 0.4575930 0.8192771
bartmachine  0.2580645 0.7943182 0.7419355 0.4168457 0.8095238
party        0.2741935 0.7647727 0.7258065 0.3615022 0.8131868
GLM          0.1935484 0.8079545 0.8064516 0.5871472 0.8461538
PamR         0.2096774 0.8034091 0.7903226 0.5307135 0.8433735
nnet         0.2258065 0.8096591 0.7741935 0.5179397 0.8205128
SVM          0.2580645 0.8329545 0.7419355 0.4363636 0.8000000
gboris commented 8 years ago

apart from bartmachine all access the set.seed function

bartmachine: seed =
Optional: sets the seed in both R and Java. Default is NULL which does not set the seed in R nor Java.

zacdav commented 8 years ago

close if fixed by new commit