search

neheller / kits19

The official repository of the 2019 Kidney and Kidney Tumor Segmentation Challenge
Other
429 stars 179 forks source link

Statistical comparisons for top solutions #37

Closed JunMa11 closed 5 years ago

JunMa11 commented 5 years ago

Dear @neheller,

For the ranking scheme,

Teams will be ranked by their average score per case, with the highest being the winner.

Would it be possible to add pairwise statistical comparisons between top solutions as BraTS, Decathlon and iSeg did?

This may help to identify the real best solution because a small performance improvement can't indicate particular dominance of a method over the others.

Thanks for organizing the great challenge, and I really enjoy it. Looking forward to your reply!

Best, Jun

neheller commented 5 years ago

Hi @JunMa11,

Yes, we're going to use the bootstrap for this. We'll describe the method in some detail when we release the leaderboard, but we plan to follow what's described in this paper: https://arxiv.org/abs/1806.02051

Cheers, Nick

JunMa11 commented 5 years ago

Hi @neheller ,

Thank you very much for the quick reply. I'm so happy that you have considered this. KiTS is really one of the best organized challenges. Again, thank you and all other organizers.

Best, Jun