search

crowdAI / crowdai

Fighting for Open Science with Open Data
https://www.crowdai.org
GNU Affero General Public License v3.0
149 stars 32 forks source link

simple fix for baseline leaderboard precision in view layer, does not alter DB #1091

Closed natejenkins closed 5 years ago

natejenkins commented 5 years ago

The root of the problem is in how the baseline rows are inserted into the Leaderboard. In calculate_leaderboard_service.rb#create_leaderboard, score and secondary_score are set to submission.score_display and submission.secondary_score_display respectively. However for a baseline those same values are set to submission.score and submission.secondary_score. The score_display values are calculated in truncate_scores, which operates over all submissions with the appropriate challenge_round_id, so it should be safe to use those values with the baseline submissions, but it is unclear to me if that is desired behavior.

In this PR it only fixes the display of the baseline score to account for the precision in the challenge round. It does add one minor tweak so that all scores will show N digits of precision instead of the current behavior where for example 1.5 shows as 1.5 instead of 1.500 for N=3. leaderboard

natejenkins commented 5 years ago

Normally I think this should be extended to also cover the display of the secondary scores and I would be happy to add that to this PR.

natejenkins commented 5 years ago

Moving to https://github.com/AICrowd/aicrowd