Multi target evaluation example

[vutle]

Hi everyone, I'm using version 1.9.1 of Meka from Maven.

I tried using multi target using the same code as multi label example in Github for train, cross validation and evaluate test set. In multi label we have access to the evaluation function below:

//Cross validation meka.classifiers.multilabel.Evaluation.cvModel(classifier, train, NoFolds, "PCut3", vop);

//Evaluate test meka.classifiers.multilabel.Evaluation.evaluateModel(classifier, train, test, "PCut3", vop);

Question: Can I cast the multi target classifier to multi label and use the same function?

MultiTargetClassifier classifier = new meka.classifiers.multitarget.BCC();

result = meka.classifiers.multilabel.Evaluation.evaluateModel((MultiLabelClassifier) classifier, train, test, "PCut3", vop);

or? result = meka.classifiers.multilabel.Evaluation.cvModel((MultiLabelClassifier) classifier, train, NoFolds, "PCut3", vop);

If I use the above functions with cast to multilabelclassifer, I get different results to the Explorer. E.g. below:

---

| Java result

== Evaluation Info

Classifier meka.classifiers.multitarget.BCC Options [-X, Ibf, -S, 0, -W, weka.classifiers.trees.J48, --, -C, 0.25, -M, 2] Additional Info [0, 2, 1] Dataset TrainSet Number of labels (L) 3 Type MT-CV Verbosity 6

== Predictive Performance

N(test) 2384 L 3 Hamming score 0.859 Exact match 0.587 Hamming loss 0.141 ZeroOne loss 0.413 Levenshtein distance 0.141 Label indices [ 0 1 2 ] Accuracy (per label) [ 0.988 0.988 0.599 ]

---

| Explorer Results

== Evaluation Info

Classifier meka.classifiers.multitarget.BCC Options [-X, Ibf, -S, 0, -W, weka.classifiers.trees.J48, --, -C, 0.25, -M, 2] Additional Info [0, 2, 1] Dataset MultiLabel Number of labels (L) 3 Type MT-CV Verbosity 3

== Predictive Performance

N(test) 2384 L 3 Hamming score 0.999 Exact match 0.998 Hamming loss 0.001 ZeroOne loss 0.002 Levenshtein distance 0.001 Label indices [ 0 1 2 ] Accuracy (per label) [ 1.000 1.000 0.998 ]

---

Is there similar evaluation functions for cross validation and evaluate test data using training and test instances?

Please show me how can I use those Evaluate functions in Multi Target classification.

Thanks

Le

[fracpete]

See thread on mailing list: https://sourceforge.net/p/meka/mailman/message/36277584/

[jmread]

To summarize here: It seems that randomization is done by default in the explorer, but you can change this with the Randomize button in the Options of the explorer.