search

Waikato / meka

Multi-label classifiers and evaluation procedures using the Weka machine learning framework.
http://waikato.github.io/meka/
GNU General Public License v3.0
200 stars 76 forks source link

Threshold in MEKA explorer #58

Open JessicaKuo opened 5 years ago

JessicaKuo commented 5 years ago

Hello,

I want to use MEKA for multi-label classification by using ANN as classifier, so I choose DBPNN with BPNN,like the figure shows:

image

And, I want to use Rcut (top-5 ranking) as threshold ,but the default in the explorer is Pcut1, so I want to ask that how can I change the threshold in evaluation options for top-5 ranking ?

Thanks in advance!

jmread commented 5 years ago

Hi,

RCut (fixed cardinality for all test examples) is not implemented in Meka. But Meka can give you the confidence output of all labels per test instance, so you can simply threshold and set the top 5 labels to 1 (and the rest to 0). Out of curiosity, why do you want to use Rcut in particular?

JessicaKuo commented 5 years ago

@jmread

Thanks for your instant reply!

The reason why I want to used Rcut is that each instance can be assigned up to 5 labels in my dataset , and I also want to try different threshold as evaluation. So , May I have a question that is there any other thresholds methods implemented in MEKA except Pcut and Rcut?

Besides, How can I save the confidence output of all labels from MEKA and the corresponded answer of each testing instance?

Thanks for your patient looking!

jmread commented 5 years ago

In the Meka GUI, you simply need to increase the level of 'verbosity', then you can see the different confidence outputs for each label and each instance. These confidence outputs are also available in the evaluation code of Meka. Of course, it depends on having a base classifier which is able to output confidences.