AUROC and dataset

[atefeh-f]

Hello , I have 2 questions. Do you have any dataset similar toy dataset? Because i need at any dataset similar toy dataset. When i am runing deepchrome with toy dataset then final output is:

==> time to learn 1 sample = 2.8497934341431ms
ConfusionMatrix: [[ 5 0] 100.000% [class: 1] [ 0 5]] 100.000% [class: 2]

• average row correct: 100%
• average rowUcol correct (VOC measure): 100%
• global correct: 100%
• AUROC: 1 ==> saving model to /home/msfathalian/deepc/code/results/toy/model.99.net
  ==> testing on test set:
  [=================== 10/10 ===================>] Tot: 8ms | Step: 0ms

==> time to test 1 sample = 0.88992118835449ms
ConfusionMatrix: [[ 1 1] 50.000% [class: 1] [ 8 0]] 0.000% [class: 2]

• average row correct: 25%
• average rowUcol correct (VOC measure): 5.0000000745058%
• global correct: 10%
• AUROC: 0.4375

Auroc is very low! (0.4375). Why? Best Regards

[rs3zz]

The reason AUC score is so low is that the toy dataset has only ~10 samples so the model is not able to train well on them.

We have provided details on generating data similar to toy dataset in the paper as well as README. With larger training samples, the AUC score will improve.