Break up "Best Practice" solution vote category

[Voileexperiments]

Currently there are two category of solution votes: Best Practice and Clever.

The problem is, this is like if Oscar award only has the "Best Movie" category. There are many categories for any award for obvious reasons: there are many meaningful aspects of qualities among a work (or a piece of code). And I think it's best to inform everyone about the various aspects of any solution.

Best practice can really mean at least one of these qualities:

• fastest code
• shortest code
• uses least memory
• simplest code
• easiest to understand
• most concise
• most informative comments
• easiest and fastest to implement
• most versatile
• most thorough (e.g freeing all memory after use to prevent memory leak, intensive input validation)
• use existing wheels and not inventing one again

Very often there is a tradeoff between these qualities (e.g space-time tradeoff, using simpler algorithm when allowed, etc). Best Practice is just clumping all of them together as an amalgam of nothing, so it's not very helpful as a whole.

Also, is Clever vote actually meaningful? The primary use for clever vote is cough cough a tool of insult when people bitchingcomplaining about solutions they don't like, in the form of "this is clever, not best practice" (aka #1390). Otherwise I can't recall any situations where it's actually doing anything (assuming Best Practice is split up).

(Incidentally, since code_challenge.label_vote.labels is an array anyway, I guess it's supposed to have multiple choices in the first place? Then breaking down Best Practice would certainly be the way to go.)

---

Related: #1390

┆Issue is synchronized with this Clickup by Unito

[anter69]

I suggest to keep it simple. If you break it down to more possibilities, people get confused and just click something. How would Average Joe tell if it's the fastest code? Does he know all the possibilities? Would he measure them? Or someone else?

This could probably work if there would be some kind of built-in metric in CW, that would measure memory usage, runtime, etc of each solution, and would make a ranking based on that. Although that could still be misleading (think of random tests, execution time on different servers, etc).

I would even go in the other direction and suggest that instead of "best practice" and "clever" have only one single upvote button. This would make the best/shortest/funniest/most readable/etc. solutions rise to the top.

Or you could implement an upvote/downvote button (like reddit), and then you can show "best" and "controversial", if you like.

[Voileexperiments]

IIRC CW in its oldest days only have one upvote button (you can see this from the images on CW main page when you're not logged in also they should be updated). I guess that enough people complaining about most upvoted solutions that they don't like (aka #1390) made the introduction of "clever" category, because otherwise there's not much incentive to open more categories.

Which means clearly clever category are not doing their job of "shutting people who complain about solutions they don't like's mouth" properly. Also what clever means is really vague (it's kind of like "funny" votes on Steam reviews, which I still don't know what they're supposed to even represent at now), so it's not functioning as a category that is meaningful.

[anter69]

In my understanding "clever" means unusual or out-of-the-box, but I agree it's different for everyone. But the same stands for "best practice": how would a newbie tell if a given code snippet is best practice? Is there even a best practice? Maybe google's best practices are different from facebook's...

However, a single upvote button - without any special naming - could be used as "like", "funny", "clever", etc.

[leerssej]

how would a newbie tell if a given code snippet is best practice?

Why are judgments of 'best practices' solicited from folks with (definitively) little to no expertise?

[jhoffner]

Why are judgments of 'best practices' solicited from folks with (definitively) little to no expertise?

How can you possibly "definitely" know that?

[leerssej]

new·bie noun informal noun: newbie; plural noun: newbies an inexperienced newcomer to a particular activity - https://en.wikipedia.org/wiki/Newbie

[jhoffner]

@leerssej so you are telling me that if Linus Torvalds signs up today on CW as a 8 Kyu "newbie", that he "definitely" has little to no expertise?

[leerssej]

I can't tell if they are Linus Torvalds, or not. (or even if it were him, just how good his PowerShell skills might be....) Unfortunately, I am certain that I have no proof that a newcomer has the expertise to weigh in on a decision about a 'best practice'. I am just wondering if the soundness of the 'best practices' metric might be improved by having only folks with demonstrated skills be allowed to make judgments in this category?

[jhoffner]

That I agree with. That there is no "proof". I was just amused by the "definitely" part ;)

Before this thread I had actually forgotten that the best practices vote wasn't a privilege, which I agree that it should be. IDK if clever should be allowed to be voted on without the ability to vote on best practices - probably not a good idea. I also don't know at what level we can really consider someone proven enough, since if we really wanted to solve this problem we would have to make the required honor pretty high, and even then that won't fully solve the problem (but would alleviate pressure).

75 honor is an open spot in the privileges chart that is low enough that we don't completely kill the feature from being used often, but high enough that we at least ensure some level of competency - though it's still a low bar.

Ideally some sort of Hacker News machine learning/complex algorithm would be put into place to weight the votes by who is voting, but that's not realistically going to happen anytime soon.

[kazk]

Related: #777 Make "Best Practice" voting an honor or rank based ability

[leerssej]

I really like your HackerRank idea! Hoping it could somehow be easy to implement. Maybe a Sensei classification for ranking some of the lower kata's solutions? (I don't mean to be unkind, but currently it almost seems like 'best practices' really might be taken to mean 'most popular' - at least in the lower javascript katas.)

full disclosure: I am a complete newb here working on picking up a new language. I've been struggling to grasp just how acceptable many of the top scoring best practices will be if I were to try to use them in production code.

Also, thank you for putting all this together! It is an incredibly engaging site and a really enjoyable way to work on improving my skills.

[Voileexperiments]

I've been struggling to grasp just how acceptable many of the top scoring best practices will be if I were to try to use them in production code.

That's a straw argument though, "production code" is not a valid (or proper) context to judge how the code should be written. You need to consider more specific scenarios.

Also, this goes without saying, but the most upvoted solution is usually most upvoted by being the oldest solution, so really, it's not very meaningful.

[leerssej]

@Voileexperiments Hmm. Your comment above makes me like your initial suggestion even more. As someone with rudimentary skills, I would find it particularly helpful to have additional objective rankings to help guide me in adjusting my style.

I had been wondering how much the solution rankings might have changed over time: great to know. Thank you!

[Voileexperiments]

@jhoffner By the way, I already have a setup (elasticsearch+kibana) to collect all CW webhook events (I just haven't started collecting data with it yet), so I can probably collect the data for a week or so and see the distribution of voter honor for all solution votes that happened. Then we might be able to see the nature of the situation at hand.

[jhoffner]

@Voileexperiments thats sounds great, except I already started to disable global events for users, per your logged issue - so that might not work for long.

For the objective rankings, I don't know that they can ever be truly objective, but I think adding a code review feature instead of an up-vote (or keep votes but simplify them to something along the lines of "I like it") makes sense.

We built a code review feature for Qualified which is along the lines of what I believe is being discussed here. Screenshot:

In addition, users do self-report their skill level (and are usually pretty honest about it), so that can be taken into account for weighing the up vote score potentially (if we keep up votes).

[leerssej]

I really don't want to cause offense or ill will here, but as a very new user trying to glean feedback and direction from the solutions and the aggregated feedback, I am completely at a loss for what to conclude from what I am seeing in the solution rankings now. If I had to be honest I would say I currently see a lot of gaming of the system, though I am hoping that this is just at the lower levels. For example: Hardcoded lookups as the 4th ranked 'Best Practice' here. I know this is anecdotal but I have to really wonder what is going on with all the folks in the comments supporting this as an effective practice in a coding challenge. Additionally in just about half of the solution sets I have tried to parse there are quite a lot of 😏 solutions at or near the top of many of the rankings. (and other folks expressing much disbelief in the comments at how solutions employing such ham handed approaches could have ever come to be so well received.) It really seems like @Voileexperiments is on to a great idea with this issue's initial suggestion. Generating objective measures is probably going to be the most immediate way to quickly provide some helpful/useful rankings to users to begin to gauge the quality of their solutions. Relative to the self ranking above, as a former teacher, I would take those with a grain of salt if there isn't a tacit understanding that an expert is going to review those rankings and judge the creator harshly if they were to try and fudge their self assessment.

tldr: Dunning-Kruger Effect

[Voileexperiments]

@jhoffner Well, I guess what I mean is that if someone like me can finish a setup capable of doing data analytics easily (I finished it in a day), surely it can be done officially with similar effort? Besides things like this, I think lots of people would want the stats.

@leerssej

Hardcoded lookups as the 4th ranked 'Best Practice' here.

You see, that's why you need to learn more ;-) Hardcoding (the more official term is "lookup table") is a perfectly valid technique used in many places where memory/space requirement is not strict (or making a lookup table would not cost you lots of memory/space in the first place) but you need that performance boost that can't be achieved otherwise. e.g CRC32 implementations use lookup table all the time (or Rainbow tables). Or in this case, since there are only a few known candidates given the input range anyway, just making a lookup table is enough. Unless you change the input range, it's always be valid ;-)

(Also, if that kind of solution works on a kata where it shouldn't be possible, the tests are badly written and the kata author/translator should be blamed instead.)

---

Speaking of which, the current tooltip for Best Practice and Clever votes are as follows:

Vote this solution as a best practice. Best practice solutions tend to be a good balance of performance, readability and maintainability. Vote this solution as being clever. Clever solutions tend to be very creative or make use of obscure language features. They usually are not code that you would want to put into production.

Which is very misleading, especially for Clever votes, the whole tooltip is ridiculous:

• very creative: I assume what this means is "taking advantage of the nature of the problem". Then isn't it best practice? After all, you need look at the input range to decide the algorithm to be used, that's like the basic skills every competitive coder (or coder, even) needs to have. And isn't being clever part of best practice? Or do I have to be dumb enough to code everything the hardcore way to become best practice? (No joke, I've seen these kind of votes in new, beta katas)
• make use of obscure language features: "Obscure" is the wrong word to be considered; "appropriate" would be the correct one. It also doesn't help that lots of people, especially newbies, don't even know the built-ins their language have, which leads to C-like code everywhere (you don't need to know a damn about built-ins when you only use things as advanced as arrays at most!).
• They usually are not code that you would want to put into production: Yes, this is a metaphor; it doesn't stop people taking it seriously anyway. Also "production code" is a false premise: it is not a valid context, so it's meaningless to think about what code would be most suitable as "production code".

The tooltip and usage of Clever votes is what make me think that Clever vote was something that was conjured up all in a sudden after people's complaining about golfing solutions. Well, if an entire category is created out of people complaining about something, how'd you expect people to use that for? ;-)

[jhoffner]

Well, I guess what I mean is that if someone like me can finish a setup capable of doing data analytics easily (I finished it in a day), surely it can be done officially with similar effort?

We actually use ES for analyzing much of Codewar's data already, though we aren't tracking honor changes as of now. Though storing the data in ES is just part of it, you need to design how a stats feature would work, determine what security needs to happen, and then implement a UI to show the data, then test and document, then deal with maintenance and people who freak out if one bit of data is off. Meanwhile thats just one "nice to have" feature added to a TODO list a mile long.

The tooltip and usage of Clever votes is what make me think that Clever vote was something that was conjured up all in a sudden after people's complaining about golfing solutions

This is 100% correct. We were like, crap there are all of these clever but "I would fire you if you ever did this in production" type solutions being created and after some deliberation on what words would help divide good real-world solutions vs "I'm simply coding this for fun", thats the pair of terms we landed on. Its not perfect, and people argue... but its the internet so... people are going to argue.

We even used to allow both votes at the same time - but then we switched it to one or the other because otherwise it didn't really help. And it did help. Its not perfect and we see flaws of course, but it definitely has helped vs not having the dual voting choice at all.

It was also devised earlier in CW's life when we had still hoped that the primary content found on CW would be more real-world based, and not a bunch of clever little puzzles - so best practices would have made more sense in "context'. To your point, a lot of these solutions are being written for challenges which don't have a real-world context to begin with so I can see why it may seem weird to call them best practices in many cases. Codewars has always been dev resource starved, and failed to ever provide guidance on content in a way that would direct authors towards producing real world challenges that were also fun to do. People build what they want to build and thats fine too.

In the short term, if you want to provide better suggestions for how to improve the tooltips, I'm all ears.

[Voileexperiments]

It was also devised earlier in CW's life when we had still hoped that the primary content found on CW would be more real-world based, and not a bunch of clever little puzzles - so best practices would have made more sense in "context'. To your point, a lot of these solutions are being written for challenges which don't have a real-world context to begin with so I can see why it may seem weird to call them best practices in many cases. Codewars has always been dev resource starved, and failed to ever provide guidance on content in a way that would direct authors towards producing real world challenges that were also fun to do. People build what they want to build and thats fine too.

Real-world based content? Topcoder already has tons of that. At least I don't see how that could possibly be implemented given that the only available tool all along is automatic testing (also with the focus of TDD, you've pushed yourself into the corner of "anything that I can't test is good to go"). And then most of the katas are basically copying problems from other entry-level competitive programming sites anyway, so if anything the move away from being just like every other site like those is actually the more surprising thing; that there are people actually spending time crafting katas that are actually original and novel.

But I think now CW fills a very specific and important niche that literally every other site does not cover at all: CW is the only place where one can learn the basic language feature and explore the advanced language features. aryan-firouzian has made lots of C# katas that taught us lots of seldom used but very important fundamental language features. Meanwhile, most of the "clever little puzzles" you're talking about is not actually "clever little puzzles" (well, except golf katas): they require a firm grasp of the language to even be tackled.

Yes, it's not the "real-world based content" everyone was anticipating in the first place. But they're also to the point, and teaching advanced language features better than them. I learn more from those katas than doing the old "real-world based" katas since the latter is mostly just "composing a bunch of 6kyu stuff, no brain required". (It's also why everyone thinks early katas are overrated.) Everyone can compose a bunch of 6kyu stuff; what's harder is to get a firm grasp of the language one's using, and I think CW's track record is showing that it's still the shortest plank on the bucket even after myjinxin2015 and donaldsebleung has written a whole series of JS/PHP tutorial series. Without a firm grasp you can still compose a bunch of 6kyu stuff; they'd just be crap and spaghetti.

Also, CW is the only place where other concepts can be introduced, e.g statistics, numerical computation (hint: it's a big field), security/crypto, formal proof, machine learning ... . Are they "real-world based"? Not if it's the "real-world" you (or most of the early people)'re implying, but that's a very narrow and myopic view. And I find these katas much more fun because I actually learn new things from them.

[Voileexperiments]

In the short term, if you want to provide better suggestions for how to improve the tooltips, I'm all ears.

It's more about a long-term solution rather than a short-term one. Lots of things have been changed during the years (and probably in a good way), including:

• now everyone's expected to write tests that can actually enforce the specs they wanted, and if solutions passed in an unintended way, it's author/translator's fault and they have to fix them, not the solver is "cheating" or anything
• now solution ranking are based on "the difficulty of conjuring up a solution", not "the difficulty of the subject at hand"
• now most, if all power users know that Best Practice is not the Best Practice many years ago, and dumbifying code does not constitute good practice (and Clever vote is mostly our way of saying "this solution was written for fun, not a serious attempt")
• now it's much easier to fix things (except with all the publish timeouts, but that's a software issue, not a bureaucratic issue)
• now it's expected that "my code does not work" is not an issue
• now we can retire bad beta katas

So it might look impossible to change the voting mechanism now, but soon enough.

Meanwhile, I guess the focus of Clever vote should be focused on "having fun and not a serious attempt" more than the current ones which is blatantly anti-golf and, as a result, anti-"advanced language features".

[jhoffner]

I learn more from those katas than doing the old "real-world based" katas since the latter is mostly just "composing a bunch of 6kyu stuff, no brain required".

Yeah my point wasn't that the non-real world kata don't teach you anything, and not that we should have a bunch of trivial everyday problems (that most of us are probably already doing anyway), but simply framing the challenges within a real world context more often. For example lot of times you learn a trick because the kata set specific constraints that guided you to a "clever" solution (like say utilizing bitwise operators), however the constraints seem unrealistic in that you could get away creating a solution like that in the real world, because you couldn't rely on such specific constraints being guaranteed. However there might be a real world example where those constraints would be there - they just aren't obvious. So then you end up using some advanced language feature but then also feeling like "in what context am I actually going to use this in the real world"?

So in other words, if the setup of the kata helped not just demonstrate how you could utilize advanced language features, but also provided a realistic context (or at least a framing of it) in which it could be used in production code, then a developer would be educated more than if done without a real world context.

With that said, its really hard to come up with interesting kata, that also guides (or at least permits) you towards an advanced solution, that is also framed within a real world application of the usage. We are focusing on these types of challenges/contexts for Qualified's library and its a battle for sure.

It's more about a long-term solution rather than a short-term one.

I just meant identifying a quick win for now, because resource/roadmap wise, starting on the long term solution is a bit of a ways out.

[fluffy-hamster]

just to throw an idea out there but maybe a "I learnt something" button may be interesting?

There is some discussion here about clever not being meaningful and "best practice" not being, well 'best practice'.

But, a "I learnt something vote" would be (hopefully) a meaningful vote that all users could participate in. The side-effect might be that the top-solutions ending being the most educational ones. Which maybe more interesting as a metric and may further encourage users to write 'helpful' (e.g well-documented) solutions.

This suggestion really strikes at the purpose of solution voting; maybe instead of trying to figure out ways to upvote "good" (whatever that actually means) solutions, maybe the emphasis is misplaced and instead CW should focus on encouraging "educational" solutions.

Notice also that "educational" solutions may in plenty of cases actually end up mirroring best practice, but (hopefully) with less ambiguity.

[anter69]

"I learnt something" button

I think that's the best idea so far! But if this should be a new, additional button, or a replacement of the current ones, that's a bit more difficult question.

[nomennescio]

What about a button "This is beautiful!" ?

I know I know beautiful code when I see it. Usually production code is not beautiful code, as you need to code for it to be maintained even by junior developers, or it was developed quickly to just get the job done.

[Voileexperiments]

Usually production code is not beautiful code, as you need to code for it to be maintained even by junior developers, or it was developed quickly to just get the job done.

This is the exact thing I was objecting to in #1390 and here: the whole narrative is wrong and misleading. That one single sentence you said there is already filled with the contradictory ideas that results in this narrative:

• using "clever" to actually mean "code that I don't like"
• inflating the idea of junior developers and enforcing all code to be C-like or Java-like (even when you're writing code in languages like Python/Ruby) to amplify the stereotype that junior developers can only read imperative code
• defining maintenance as not about being versatile and robust but being dumb and imperative (which would be very difficult to maintain, to the contrary)
• lowering the bar of entry of code not by providing insightful comments but instead dumbifying the code itself

Yes, I can go on on this like forever, because they're very detrimental to people who are learning, and is overall harming CW as a learning platform. Writing and delivering code is one thing, not holding completely wrong and toxic ideas/mindset is another; much more WTFs come from people having the wrong ideas than from writing plain bad code.

[nomennescio]

It's up to CW to decide how much and to what extent it wants to be a learning platform. From experience I can say I've learned a lot by studying code of others who I knew were excellent craftsmen. And I've come up with some inventions of my own through years of experience. I always enjoy studying solutions of others once I've submitted my Kata, and I would advice others to do the same. If you want to specifically point people to great code, why not write about it, as you clearly have some passion for it?

And to repeat, each 'button' CW can add, can and will be misused. I do not think that people will consistently review all submitted code and vote for the 'best' candidate, irrespective of what criteria would be applied (most practical, clever, beautiful, I've learned something etc.).