search

woven-visionai / wts-dataset

25 stars 1 forks source link

Bug in evaluation code #7

Closed trinhxuankhai closed 6 months ago

trinhxuankhai commented 6 months ago

From lines 143 to 146 of code evaluation/eval-metrics-AIC-Track2/metrics_all.py, if I do not submit the result to a specific sample, this section of code states that that sample will count zero, but it should also increment the num_segments_overall, or that sample will be skipped and the evaluation code can be bypassed by submitting only serveral sample with the best score. image

woven-visionai commented 6 months ago

Hello, thanks for pointing out this issue. We have already fixed it in our latest master. Please take another look.

trinhxuankhai commented 6 months ago

Will the evaluation system on the leaderboard be updated following the issue?