toonvds / CostSensitiveLearning

Code for the paper "Predict-then-optimize or predict-and-optimize? An empirical evaluation of cost-sensitive learning strategies".
MIT License
11 stars 2 forks source link

Statistical significance tests #3

Closed lily706 closed 5 days ago

lily706 commented 1 week ago

Hi,

Can you please share your code for the critical difference diagrams and the statistical tests you used? I'm still a beginner and not sure how to do that and didn't find some code online.

Thanks a million, it would help a lot!

toonvds commented 1 week ago

Hi!

I used the "compute_CD" and "graph_ranks" functions from Orange for the critical difference diagrams, and the "wilcoxon" and "friedmanchisquare" functions from scipy for additional testing.

I hope this helps!

lily706 commented 1 week ago

Hi! It does, I'll try to implement it and see how it goes, thanks a lot! Can you please tell me where you used Wilcoxon exactly in your experiments?

lily706 commented 1 week ago

Hi again,

I tried testing Orange but it looks like it's deprecated, the functions no longer exist in Orange3. Do you mind sharing the version you used?

Can you also tell me where you used Wilcoxon test? And which test did you use to determine the values in bold and italic in Table 9 of your paper?

Much appreciated!

toonvds commented 5 days ago

The Orange version was the following Orange3==3.27.1, see also the requirements file.

The Wilcoxon test was used for the results in the tables for a pairwise comparison with the best result.

lily706 commented 5 days ago

Thanks a lot!