Open piyaliG opened 7 years ago
I am also using the default model given in this repo and evaluating the default evaluation dataset, but mine gives accuracy of 0.711933. I am curious why the evaluation accuracy differs if we are using the same default model and dataset.
I also get 0.711933 overall accuracy and the following confusion matrix. Is this the right result?
@simonwsw many thanks for publishing the evaluation code. More researchers should do the same! Stared your project!
[eval] accuracy 0.711933
label 01: 58 [ 58 00 07 24 05 00 00 04 00 00 01 00 00 ]
label 02: 96 [ 00 96 00 01 01 00 00 02 01 00 00 00 00 ]
label 03: 62 [ 04 06 62 17 03 00 00 07 01 00 00 00 00 ]
label 04: 73 [ 13 01 06 73 03 00 00 01 02 00 02 00 00 ]
label 05: 80 [ 06 02 03 03 80 00 00 00 05 01 01 00 00 ]
label 06: 43 [ 17 01 09 04 03 43 10 01 01 01 07 00 03 ]
label 07: 98 [ 00 00 00 00 00 00 98 01 00 00 00 00 00 ]
label 08: 91 [ 01 01 01 05 01 00 00 91 01 00 00 00 00 ]
label 09: 93 [ 01 02 00 02 01 00 00 01 93 00 00 00 00 ]
label 10: 64 [ 04 18 03 01 04 00 00 00 03 64 00 00 02 ]
label 11: 25 [ 09 00 01 37 02 00 00 25 00 00 25 00 00 ]
label 12: --
label 13: 00 [ 100 00 00 00 00 00 00 00 00 00 00 00 00 ]
Finished
Hi, I am trying to run the default evaluation of the network. I am getting the following output:
[eval] data with 1364 seq [net] loading model uni_image_np_50.t7 nn.Sequencer @ nn.Recursor @ nn.MaskZero @ nn.Sequential { [input -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> (7) -> (8) -> (9) -> (10) -> (11) -> (12) -> (13) -> (14) -> (15) -> (16) -> (17) -> (18) -> (19) -> (20) -> (21) -> output] (1): cudnn.SpatialConvolution(4 -> 32, 3x3, 2,2) (2): nn.SpatialBatchNormalization (4D) (32) (3): cudnn.ReLU (4): cudnn.SpatialConvolution(32 -> 64, 3x3, 2,2) (5): nn.SpatialBatchNormalization (4D) (64) (6): cudnn.ReLU (7): nn.SpatialDropout(0.400000) (8): cudnn.SpatialConvolution(64 -> 128, 3x3, 2,2) (9): nn.SpatialBatchNormalization (4D) (128) (10): cudnn.ReLU (11): nn.SpatialDropout(0.400000) (12): nn.Reshape(1152) (13): nn.Linear(1152 -> 512) (14): nn.BatchNormalization (2D) (512) (15): cudnn.ReLU (16): nn.Dropout(0.5, busy) (17): nn.Linear(512 -> 512) (18): nn.LSTM(512 -> 512) (19): nn.Dropout(0.5, busy) (20): nn.Linear(512 -> 13) (21): cudnn.LogSoftMax } [eval] accuracy 0.502874 label 01: 11 [ 11 39 15 27 06 00 00 02 01 00 00 00 00 ] label 02: 97 [ 00 97 01 01 00 00 00 01 00 00 00 00 00 ] label 03: 16 [ 00 67 16 07 02 00 00 01 07 00 00 00 00 ] label 04: 37 [ 00 13 02 37 01 00 00 04 43 00 00 00 00 ] label 05: 70 [ 01 08 02 04 70 00 01 02 12 00 00 00 00 ] label 06: 14 [ 01 03 00 05 00 14 13 06 58 00 00 00 00 ] label 07: 97 [ 00 00 00 01 00 00 97 02 00 00 00 00 00 ] label 08: 86 [ 00 03 02 03 01 00 00 86 03 00 00 00 00 ] label 09: 95 [ 00 02 00 01 00 00 00 01 95 00 00 00 00 ] label 10: 27 [ 00 41 00 03 01 00 01 01 25 27 00 00 00 ] label 11: 00 [ 02 33 18 42 01 00 00 03 01 00 00 00 00 ] label 12: -- label 13: 00 [ 100 00 00 00 00 00 00 00 00 00 00 00 00 ] Finished
I am curious to know why the evaluation accuracy I recieve using the default code base is very much off compared to the expected. Any pointers would be very helpful!
Thanks and Regards, Piyali