Rating in CODE & F2F

[ThomasKranitsas]

Why we don't have rating in CODE & F2F? Those are the most commonly used tracks under DEVELOP.

Also if you check the leaderboard in https://www.topcoder.com/community/statistics/ you will see that all members there are red rated with wins in DESIGN & DEVELOPMENT. Both of those tracks seems to be obsolete and since we don't have rating for the current used tracks it's impossible to update the leaderboard.

9/10 members from the leaderboard are inactive the last few years and I don't see why we should not update that leaderboard.

[vic-tian]

We are currently discussing the member metrics for the community internally. One of the important differentiators is active vs. inactive member. Once we have a better structure around those we would employ them to update the scoring and build better leaderboards.

@ThomasKranitsas: Talking about ratings, what would be the parameters that are the most important for a TC member? Are there things that we haven't thought about, or missing current?

[MonicaMuranyi]

I think the most important metrics at the moment are the number of won challenges and the winning rate in a given period. The rating would also be a good metric but we don't have it in Code and F2F, as Thomas mentioned. The all-time leaderboard we currently have is not really relevant as those members are not active in years. Maybe we can update the leaderboard to only consider the results in the previous year.

[birdofpreyru]

Yeah, the winning rate should be it. Should count second places. Should be weighted by time and, probably, by earned money (i.e. a victory/loss an year ago has a way lower impact on the ranking than a recent one; and a victory in a contest with big prize has more weight that a victory in a contest with a small prize).

[birdofpreyru]

I'd suggest to calculate the current rank of a member, R by the formula: R = sum_i (prize_i * exp(-t_i / tau)) Where sum_i is the sum over all his past challenges, prize_i is the money earned in the i-th challenge, t_i is how long time ago that challenge took place, and tau is the weight factor, which tunes how fast we forget about past challenges.

[MonicaMuranyi]

I would vote for having a new leaderboard each year as opposed to having an all-time leaderboard with lower impact for older wins. But I agree the prize should also be considered.

[birdofpreyru]

Why?

Do take into account that with the leaderboard based on earnings in a year, you basically makes everyone's annual earnings public. With the formula I propose, it is a way trickier to estimate someone's annual income based on the resulting number.

[MonicaMuranyi]

If you consider the challenge date in the ranking formula, a member who won a challenge in April will have an advantage over a member who won a challenge in March for example. I am not sure why this would make sense.

I understand the idea that more recent challenges are more relevant but that's why I proposed a leaderboard for a certain period of time (e.g. one year).

[birdofpreyru]

The date there is relative to the present day, thus a participant who has won a contest one month ago will have a bit higher rating than the other one, who has won a similar challenge two months ago - fair?

Also, it makes more sense to think about this formula in case of more active participants, when it averages results over time, giving preference to the members more active recently.

[lstkz]

@MonicaMuranyi @birdofpreyru

Here is DB dumb of the latest 7k challenges (Code, Assembly, F2F, UI prototype). https://github.com/lsentkiewicz/tc-api/tree/master/data

I suggested that you can show some example results of your formula :)

[birdofpreyru]

I will, just give me few days :)