search

comic / grand-challenge.org

A platform for end-to-end development of machine learning solutions in biomedical imaging
https://grand-challenge.org
Apache License 2.0
167 stars 49 forks source link

Scoring: exclude_from_ranking #3352

Closed chrisvanrun closed 1 month ago

chrisvanrun commented 1 month ago

Part of the tiny pitch:

This PR adds an optional (default: false) exclude_from_ranking field in the extra results columns JSON definition. Setting it to true allows additional metrics to be shown on Challenge leaderboards while being ignored while calculating the rank.

The related test is a construction copy of test_calculate_ranks since I did not find it wise to extend that test with yet another factor. It would have pushed the 12 (2x3x2) to 24 which is over the top imho.

jmsmkn commented 1 month ago

I would have thought that the template and view would need updating too as the rank will not be present for excluded fields, right?

chrisvanrun commented 1 month ago

I would have thought that the template and view would need updating too as the rank will not be present for excluded fields, right?

I might have completely missed the fact that the leaderboard templating changed if the scoring method changed! I'll look into this after the weekend.

chrisvanrun commented 1 month ago

Refactor the test(s) and have updated the leaderboard template to not decorate the excluded metrics: image