codewars / codewars.com

Issue tracker for Codewars
https://www.codewars.com
BSD 2-Clause "Simplified" License
2.09k stars 219 forks source link

Put limit on Honor earned by upvoting #1880

Open nomennescio opened 5 years ago

nomennescio commented 5 years ago

Honor should reflect mastery and relate to effort. Currently it is possible to author a low-level Kata, which will be subsequently up voted by tens of thousands Codewarriors. That would earn you tens of thousands in honor points, which does not reflect well the effort spent on authoring the Kata, nor the mastery of either language or concepts.

Therefore please put a reasonable limit on total honor points earned by upvoting, as it currently can be potentially infinite.

┆Issue is synchronized with this Clickup by Unito

nomennescio commented 5 years ago

Note that the effect of honor earned by upvoting is being reinforced by the ways Katas can be sorted; newbies to a language (the biggest group of Codewarriors) will pick 8 Kyu Katas, and a certain percentage will be selected on 'Popularity' or 'Positive Feedback', favoring Katas that already got a lot of upvotes, thereby creating even more upvotes.

Now obviously upvotes themselves serve a purpose to let the wisdom of the masses decide which katas are most enjoyable to solve. but it should not translate into potentially unlimited Honor.

nomennescio commented 5 years ago

As a suggestion, it might an idea to give Honor as a percentage of a certain fixed limit, the percentage being defined by the Popularity and/or Positive Feedback (a measure of upvotes) of the Kata.

hobovsky commented 5 years ago

Unpopular opinion: Honor Points should be rewarded only for solved kata, and nothing else.

nomennescio commented 5 years ago

Case in point: top 100 of the leaderboard has 2 authors who rank 6 Kyu, and 4 who rank 5 Kyu. For example one of these authors has finished 44 Katas, mostly of difficulty 8, 7, and 6 Kyu, and one 4 Kyu, has 2 approved translations, and authored one 8 Kyu and one 7 Kyu Kata. Those authored Katas have been solved by more than 100.000 Codewarriors, with a satisfaction of about 88%. That resulted in about 20.000 honor points, meaning that about 10% of the Codewarriors upvoted these authored Katas. That alone results in a place in the top 100 of Codewarriors.

I think that does not really reflect the essence of what it means to train and sharpen your skills, and grow in mastery accordingly.

My suggestion is to have a limit on upvotes on authored Katas, based on the level of Katas. You already earn Honor by authoring a Kata, therefore let's say you can earn a maximum of twice as much honor points by upvotes, making it worthwhile to author good quality Katas. That would give the following limits:

Activity Honor Points
Maximum honor by upvotes for authored Kata (White) 4
Maximum honor by upvotes for authored Kata (Yellow) 16  
Maximum honor by upvotes for authored Kata (Blue) 64  
Maximum honor by upvotes for authored Kata (Purple) 256

You either assign a percentage based on satisfaction rating or on percentage of upvotes, and use that to determine the actual extra honor points earned.

In the mentioned example, either 10% or 88% of a White Kata would earn 1 or 4 per Kata, for a total of 2 to 8 extra Honor points, and not the current 20.000 (and counting!)

Voileexperiments commented 5 years ago

I think the impact of authored kata upvotes dominating the leaderboard can be mitigated once the relationship is changed to something sub-linear, like honor = pow(upvotes, 0.5)? You still get more honor the more upvotes your authored katas get (a hard cap is not recommended because any kata will reach the cap given a big enough denominator) so number of upvotes still matter, but the return diminishes in the long-term. Then just adjust the 0.5 to something more balanced.

There's also, of course, the issue that users with many authored katas will win the current meta (it's very hard to get tons of katas approved nowadays but it used to be the opposite a few years ago) but I don't think it's that much of an issue compared to the one in the previous paragraph. It can be dealt with later.

Voileexperiments commented 5 years ago

Another related issue is that an upvote on a white kata is worth the same as an upvote on a purple kata, which is analogous to "getting a white kata approved gives as much honor as so for a purple kata", which wouldn't make sense at all. Naturally a purple kata has less solves than a white kata because of its difficulty. I think it's one of the fundamental reasons the authored kata leaderboard is imbalanced as it is right now.

Blind4Basics commented 5 years ago

@nomennescio : there are 3 different leaderboards, you know? ;p

The one you're interrested in (almost) already exists, it's the second tab, by "completed kata". Tho, it's only "almost" what you seek for because this one is counting the 1 honor reward per satisfaction votes given by the user after he solved a kata too. This part should be removed from the counts of this leaderboard, since it causes the same kind of advantage to users who are screening easy katas rather than those who are doing less but harder problems. In addition, that would avoid the more or less 4000 points jumps in that leaderboard, for PUs, when the "upvote database bug" occurs. x) (#1715 / see Unnamed post in #1875 too)