Wollongong: predict high+medium delay as positive

[WeiFoo]

Settings

• data : apache, duraspace, jboss, jira, moodle, spring, wso2.
• DE: np = 60 (10 times number of variable),  life = 5,
• Learner: CART and RF
• repeats: 5 times
• tuning goals: F , Precision, AUC

  Results

• Tuning goal is F raw results

• Tuning goal is PREC raw results

• Tuning goal is AUC raw results

[timm]

not clear if this a three class or a two class problem...

[WeiFoo]

Two class problem. Treat both high and medium delay as delay, and others(low delay and non-delay ) as non-delay

[timm]

can i summarize the above as "tuning rarely hurts but sometimes it does great?" e.g. the apache1 results?

and why aren't you doing 4classes? high and medium delay ow delay and non-delay )

[WeiFoo]

can i summarize the above as "tuning rarely hurts but sometimes it does great?" e.g. the apache1 results?

this statement holds for AUC and F as tuning goals and most improvements are 1,2, not that great. For precision, we got some worse results.

and why aren't you doing 4classes? high and medium delay ow delay and non-delay )

I can do 4 classes, but when we are tuning, we can only choose one type delay for one goal, say tune learner to get higher AUC for high delay. Then I think the result of baseline and tuning for high delay will be the same as #19 , where only have high delay and non-delay, a binary classification problem.