[Leaderboard] Monitor shows higher number of finished submissions than there actually are

[TjerkNan]

Describe the bug We observe for the RankPreFer challenge that there were 18 actual submissions but the monitor says we have 23.

Thought: would it be worth the investment to make some tool that can generate submissions to test monitor?

To Reproduce Observe screenshot

Expected behavior The number of finished submissions should be identical to the number of submissions we received through the SQL query we performed.

Screenshots

Desktop (please complete the following information): macos safari

[emielvdveen]

@TjerkNan The number of finished submissions should be identical to the number of submissions we received through the SQL query we performed. This is not true perse. The monitor counts events and this will never be 100% guaranteed the same. Having said that, I think in this case the monitor counts submissions events including initial submits + updates. This something we need to fix if that is the case

[TjerkNan]

• I will test your hypothesis on dev and depending on the outcome create new issues and will close this one.