search

topcoderinc / cab

9 stars 3 forks source link

Reliability bonus #50

Open lstkz opened 7 years ago

lstkz commented 7 years ago

An idea to create a new reliability system for the code track.

kondakovdmitry commented 7 years ago

One of the issues is that the current method of reliability rating calculation is quite harsh for newcomers.

For example, if you only participated in 2 challenges so far and passed review in the 1st challenge but not passed in the 2nd, then your reliability rating drops from 100% to 50% even though you only failed in one challenge. After that it takes quite a while before you reach 80% rating and become eligible to receive a bonus again even though you always pass review since that. In contrast, if you already participated in many challenges and kept passing review for the last 15 challenges in a row and then failed in one, then you loose only 1/15 and your rating drops from 100% to 93%.

Looks like it would be more natural to have the same drop for newcomers as well. I.e. the formula for rating calculation should be (1 - number_of_fails_in_last_15_challenges / 15) * 100%.