neheller / kits19

The official repository of the 2019 Kidney and Kidney Tumor Segmentation Challenge
Other
439 stars 179 forks source link

How about closing per case score of test set? #38

Closed JunMa11 closed 5 years ago

JunMa11 commented 5 years ago

Hi @neheller ,

Thanks for organizing the great challenge.

Just now, I find the per case scores of the test set are publicly available. https://kits19.grand-challenge.org/evaluation/results/ To avoid per-case finetune during post-kits, how about closing them?

Best, Jun

neheller commented 5 years ago

This is a huge problem. I didn't realize that the full json would be made publicly available with each leaderboard entry. Thank you so much for pointing this out.

I've temporarily hidden the live leaderboard and prevented new entries from appearing automatically. Unfortunately I don't think that grand-challenge.org has an elegant solution for this, and I think I'll end up having to change the evaluation script and re-upload each submission individually. I'll be in touch with the admins to confirm this, but either way it will probably take about a week. I'm really sorry for the inconvenience.

JunMa11 commented 5 years ago

Thank for your quick response. Glad that you agree with my suggestion :)