search

Waikato / meka

Multi-label classifiers and evaluation procedures using the Weka machine learning framework.
http://waikato.github.io/meka/
GNU General Public License v3.0
200 stars 76 forks source link

Single target result of J48 different to Weka #50

Open vutle opened 6 years ago

vutle commented 6 years ago

Hi I'm testing a single target using BCC with J48.
The result seems to be different compared to Weka. E.g. Accuracy is 0.525 in Meka while 0.5 in Weka for cross validation.

Is the method implement different between the two? Shouldn't I get the same answer if it is a single target?

Thanks Meka N(test) 40 L 1
Hamming score 0.525 Exact match 0.525 Hamming loss 0.475 ZeroOne loss 0.475 Levenshtein distance 0.475 Label indices [ 0 ] Accuracy (per label) [ 0.525 ]

Weka Correctly Classified Instances 20 50 % Incorrectly Classified Instances 20 50 % Kappa statistic 0.2599 Mean absolute error 0.2036 Root mean squared error 0.4321 Relative absolute error 73.8882 % Root relative squared error 117.4604 % Coverage of cases (0.95 level) 55 % Mean rel. region size (0.95 level) 31 % Total Number of Instances 40

jmread commented 6 years ago

Meka has different default parameters. For example, by default the dataset is randomized in Meka prior to cross validation, therefore the train/test sets may vary. Maybe that is the issue.