search

Waikato / wekaDeeplearning4j

Weka package for the Deeplearning4j java library
https://deeplearning.cms.waikato.ac.nz/
GNU General Public License v3.0
185 stars 202 forks source link

Predictions are inverted (sometimes) #50

Open henrygouk opened 5 years ago

henrygouk commented 5 years ago

Describe the bug I've noticed that the validation accuracy logged by dl4j is almost an exact inversion of the test set accuray reported by weka for some binary classification datasets I've been using.

To Reproduce Steps to reproduce the behavior:

  1. Download ARFF file from https://www.openml.org/d/1590
  2. Create a network in the weka explorer with three Dense layers of 100 units each, batch size of 100, everything else left as default default.
  3. See the accuracy reported by dl4j during training
  4. See the mismatch in test accuracy reported by weka

Note that this does not happen with all binary classification datasets I have used.

Expected behavior Accuracy reported by dl4j should be similar to what is reported by weka.

henrygouk commented 5 years ago

I've come up with a temporary workaround: swapping the order that the two classes are defined in the ARFF file.