search

Cloud-CV / EvalAI

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
https://eval.ai
Other
1.77k stars 787 forks source link

Add a total submissions (including failed/cancelled) limit for phases. #4348

Closed gchhablani closed 7 months ago

gchhablani commented 7 months ago

People can submit a file as many times as they want if it always keeps getting failed. We should have a daily submission limit per challenge phase. We should use max(total_limit, finished_limit) as total allowed submissions in a day, while also checking the current finished_limit.