New Judging Categories for consideration

[mikekasprzak]

Here's a thread to make suggestions for new voting categories.

The key thing to understand is that the judging categories are opinions. We can't have categories that require special knowledge. They need to be understood by all participants, be they programmers, artists, chefs, etc.

Categories Today

• Overall
• Fun
• Innovation
• Theme
• Graphics
• Audio
• Humor
• Mood

Special Categories

• Coolness
  • Determined using Magic

Categories we removed

• Technical
  • We found that the majority of users weren't able to guage how technical something was. Almost nobody looks at the source code. And thanks to the advent of Middleware, it's even harder today than it used to be.
• Community
  • We used to score people by the quality of blog posts. Ultimately it wasn't really an effective category, as the focus should be on the game.

[mikekasprzak]

Austin Allman‏ @drazil_austin

@mikekasprzak Next Ludum Dare can we add a new category to the voting? I would like to be able to vote how intuitive a game is to play. Regardless of how simple or complex a game is there are some games that just don't explain how to play and some that go out of their way to. I would like to reward the games that put in that extra effort into making sure the player knows how to play their game.

https://twitter.com/drazil_austin/status/858943574972518401

[quantumrain]

Presentation - How well put together the game is, are all the rough edges smoothed off, transitions on all the areas that need them, in-game tutorialisation, FXs on in-game actions that require them, does the game have a title, intro, endgame screen, etc

Consider splitting audio into Music and Sound Effects; many games have one or the other, and I'm more frequently seeing people use a canned music track downloaded from the internet, which then means they have to opt out of audio entirely even though they did create sound effects for the game.

[drazil100]

while I like presentation and it sounds too art and UI centered which is a huge reason I asked about an exra category but I also wanted something that covers controls and probably even the way that physics work. Intuitive is not the right name for the category either though.

Maybe polish is a better name? Polish would basically judge if the game achieved what it was trying to be and how well it managed to be it's goal. If there are weird annoying physics querks or it's unbalanced or it does a bad job explaining how to play or it's controls are just poorly mapped then it would not be a very polished game even if it is fun. That said if it goes out of it's way to make sure you not only know how to play but have it play exactly as you feel it should then even if it doesn't fit the theme or it isn't exactly your type of game and therefor not fun it can still be rewarded for it's polish.

[Jezzamonn]

Something worth noting is that the judging categories will definitely influence people's votes in the overall result, for better or worse.

Also we should try to have not too many categories because each one makes it take a little more effort to rate a game, which kinda adds up when you think about everyone's ratings. Also, the more categories there are, the less meaningful they feel. On Tue, 2 May 2017 at 15:58, drazil100 notifications@github.com wrote:

while I like presentation and it sounds too art and UI centered which is a huge reason I asked about an exra category but I also wanted something that covers controls and probably even the way that physics work. Intuitive is not the right name for the category either though.

Maybe polish is a better name? Polish would basically judge if the game achieved what it was trying to be and how well it managed to be it's goal. If there are weird annoying physics querks or it's unbalanced or it does a bad job explaining how to play or it's controls are just poorly mapped then it would not be a very polished game even if it is fun. That said if it goes out of it's way to make sure you not only know how to play but have it play exactly as you feel it should then even if it doesn't fit the theme or it isn't exactly your type of game and therefor not fun it can still be rewarded for it's polish.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ludumdare/ludumdare/issues/832#issuecomment-298504154, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ13DrezZ9Nhe_qs-EZ42eFsae2_Deazks5r1sYbgaJpZM4NM5FN .

[quantumrain]

@drazil100 I like the idea of an intuitiveness rating too- I was suggesting presentation in addition to it as POV asked for suggestions for new voting categories on twitter :)

I feel they are quite different ratings though, an intuitive game would be easy to pick up, gives the player a clear understanding of how to play, with minimal tutorialisation; a polished/well presented game has transitions, fxs, sounds, etc for everything; and I would personally put 'Feel' into a separate category for people to score the physics and general game interactions, and maybe even a forth 'Balance' category! CATEGORIES FOR EVERYONE! ;)

But moving onto @Jezzamonn's point about not having too many voting categories, I think this is just a presentation issue and we could do something like have a word cloud below the star buttons which allows people to select up to three words to describe their experience with the game, e.g.

    [x] Intuitive       [ ] Balanced        [ ] Feel
    [ ] Sound FX        [ ] Ambience        [x] Music
    [ ] Character       [x] Atmosphere      [ ] Story

Then with just three additional clicks you can highlight things in peoples games that you thought was worth focusing on, and we can list off how many clicks each got for a particular submission at the end of judging. (I don't think we should rank people on these- just leave them as informative extra stats)

Perhaps this is over thinking it though, I imagine all most people really care about (outside of the people in the top 10/100) is that they have people comment on their games?

[drazil100]

I agree with @jezzamonn. We want voting to be as easy as possible. That said maybe there should be more opt in categories. Not every game has houmor and not every game has mood but I have to judge every game for these things. I could see having and opt out of audio and into sound effects. I don't think intuitiveness or polish or presentation (whatever we decide) should be an optional category as every game SHOULD strive to be the best in said categories regardless of what limitations you face but I'm not opposed to more categories provided it doesn't bog down the judging process.

[philomory]

Here's another voice in favor of something at least akin to "polish". I somewhat agree with @quantumrain's point, insofar as it's possible for "polish" to be mistaken for "professionalism" or similar, but I think that's an issue of terminology rather than core concept. An alternative might even be "execution", as in, "You had a good idea, now how well did you execute it", except that sounds like terrible word choice to me.

Fundamentally, whatever term we choose, I really do miss having a category that encompasses the experience of interacting with the game as a piece of software. A truly fun game that's full of weird bugs doesn't necessarily deserve to be knocked down in the fun category, but what other category would you choose? Likewise, an ultimately somewhat dull game with an excellent tutorial and user interface, doesn't deserve to be bumped up in Fun, but no other category is appropriate. We need a category for that attention to detail that makes you say, "How did they manage that in 48/72 hours", as well as those unfortunate failings that make you say, "Even with only 48/72 hours, couldn't you have at least...?"

[Cerno-b]

I am with @Jezzamonn: don't use too many categories as it can get confusing.

I would even question that we need a Humor category. Maybe it's just me, but I played almost no humorous games this LD, so I opted out of that category for most of them. It's also hard to judge: What do you mean by awarding 1 star for Humor? There are jokes but they are bad? Or the game is not humorous at all? Would a horror game get 1 star or 0 stars?

Maybe we can change the way we handle the Humor category, so that you are asked whether your game is supposed to be humorous when you enter it instead of having it as opt-out? Most people seem to just leave that category in, I assume because they want as much feedback as possible. So by asking for whether a game aims to be funny, the category would gain a lot more weight. People may be more aware of the fact that if they provide humor, it will be judged on its quality, not if it's there or not.

[Jezzamonn]

Some other points to consider for a new category (or existing ones)

• Is it useful to make game creator and game voters think more about this aspect of a game? E.g. having a polish category encourages people to make more polished games, and will also probably influence ratings in the overall category.
• Is this something people strive for when they make a game? Could someone say "I want to get a really high ____ rating this LD"
• Is this a useful metric for game creators to get feedback on and try to improve on?
• Is it interesting to have category winners for this category?

Seeing as you can opt out of categories I think it's ok that categories don't apply to every game. Humor and mood are the weakest categories at the moment, in that they don't apply to every game, but I do like seeing the category winners for those games. And for some games they seem super relevant.

I personally actually like the idea of having a category to for polish, and a category for how easy the game is to understand, because hopefully they'll make people realise those are important things, and they can learn and improve on it. (And they're things a lot of games fail to do).

Maybe we could have vote on whether to add new themes on twitter or something?

[Cerno-b]

@Jezzamonn I would disagree that Mood is similar to Humor in its difficulty to evaluate, as not every game is by definition humorous, but I would argue that every game can have a good mood. Maybe the term is not perfect though. For me, Mood describes how well the game pulls off its genre believably.

Of course if you have a lush landscape with atmospheric music, that's mood. But if you make a retro game that fires up your sense of nostalgia, even if it's a fast paced arcade game, that is also mood. Or if you make a comedy game with a quirky world and characters that fit well into that world, that is mood.

The problem may just be that the term mood is generally associated with relaxed feelings and so the other two examples may not fit into that strict definition. In a sense, the Mood and Polish categories may be a bit similar. If we could find a term that captures both terms, then we could fuse them into one.

[BurgeeGames]

I would say that 'overall' should not be a voting category, but instead just an average of all of the other categories.

This might cause issues with certain opt outs, however, leaving less criteria to be averaged and giving the opt-out game a slight advantage. (suck at audio? opt out!) A modifier for opting out of a category could solve this, though I don't know what that math would look like.

I also think 'Mood' should change to 'Polish' - Mood is too hard to gauge.

Some ideas:

Combine Mood and Humor into "CHARM" Split "Graphics" into "ART" and "PRESENTATION" Split Audio into "MUSIC" and "SFX" - this was a great suggestion

Along these lines, I think there should be an option to give anonymous feedback to the developer.

People are too nice in the comments - whether it's to spare feelings or to hope for kind votes in return. Anonymous feedback is often the most valuable. I learned more from this LD by watching streamers play my game than I did from my feedback - which was filled with stuff like 'best game I've seen' - but my votes did not reflect those comments.

[ajayyy]

I like mood, it's like atmosphere. If the audio, graphics, and game play all match well, it has good mood. At least that is how I look at it.

[Jezzamonn]

I would say that 'overall' should not be a voting category, but instead just an average of all of the other categories.

This idea has come up in the past.

The problem is, how good a game is isn't directly related to how good it is in each category, it's much more subjective. For example, a game can have awful graphics, but still be a really well made and enjoyable game. Should the graphics take away from their overall rating? Up to you, as a player.

Check out this post I made once about correlations of categories to overall: http://ludumdare.com/compo/2016/01/23/ludum-dare-visualisations/ You can see that the categories roughly correlate to the overall result, but not perfectly. Seeing as it's pretty nuanced we should leave people in charge of how they think different aspects affect how good the game is as a whole.

Plus, with all this talk about what categories we should have, it's unlikely that there's ever going to be a consensus on "these are the perfect categories!" so, better to leave the overall vote unaffected by them.

I'd rather get rid of every other category than getting rid of the overall one 😉

[Cerno-b]

I'd thought about averaging the Overall category as well, but I am with @Jezzamonn. The overall rating is much more than just the sum of its parts. Look at the discussion we are having about what categories should be included at all. Summing them up to a meaningful total would be impossible to do fairly. So I'd stick with the Overall category as the single most important one.

[ajayyy]

When itch.io offered to host LD36, one of my worried was their lack of an overall rating (using an average instead), and I think it really makes it less "LDish"

[karai17]

I feel like the categories need some overhaul. All of the current categories except for humor are fine. Humor, in my opinion, shoudl just be merged in with mood. It implies that your game needs to be funny or silly to qualify and that could be very limiting.

I think more categories that give better feedback on the game itself are needed, such as a category for controls.

[ajayyy]

I don't really understand why humor is there. I guess in the end, you can easily find funny games. If a game is funny, and the audio is very upbeat, then that would be a 5 star mood too.

[local-minimum]

I think I would be for something along the lines of polish, exact wording unclear but one of the most common issues I have playing and making games are:

• understanding what to do and what to interact with
• the playability and how difficulty ramps up (will I give up on the first jump)

but also, sometimes the instructions are so long and daunting that I really don't want to even try the game out.

I think we could learn to be better game makers and have more joy playing each-others' games if we gave focus to these UX-issues.

However, the problem with the category, perhaps that the difficulty in doing these things good depend very much on how many tropes and how stereotypical you make your game. So it will be really hard for the games I fancy the most, and want to make myself, which are strange and experimental to get good votes in this category. On the other hand, perhaps that is fine too.

[drazil100]

I don't think a game has to be stereotypical in order to be ranked high in this type of category. What matters is attention to detail. Having the right sound effects or visual indicators to let you know when you have taken damage, clear easy to read UI, making sure important game objects are clear and understandable, heck even how you present the instructions on how to play is important (no walls of text).

Even experimental games can be well polished and even if they aren't they can still do well in other categories.