Investigate ways to improve game rating/commenting behavior

[mkalam-alami]

Reported facts

• One game (from a popular dev) got 5 ratings in the hours following the opening of ratings (reportedly the first 3 in ~10mn), but only one short comment
• One game despite not having links yet, has already got ratings

Concerns

• Insufficient incentives to comment on games
• Lack of play time before rating, possibly even people serial-ratings games that they didn't play (*)
• Lack of confidence by players to comment on a game that they didn't complete/felt too hard

(*) They're at least one legitimate use case for serial-rating, which is streamers playing games live but keeping ratings for later.

Things to investigate

Research to be done in the production DB:

• Whether some people have bad rating practices that should be moderated (eg. fishy rating speed)
• Whether there's a "popular dev bias" that make people have different rating/commenting behavior
• Whether there's a "hard game bias" that make people have different rating/commenting behavior

Possible fixes

• Making the comments more valuable in terms of karma
• Advertise more the comments as more valuable (possibly making the formula more public)
• Give recommandations for reviewing games (eg. any playtime >5mn is okay)
• Better moderating tools to detect/act on fishy ratings
• Additional mechanics ("karma prize"? per-entry "block ratings without comments" setting? ...) to encourage comments on games in an unbalanced ratings/comments state
• Cooldown between ratings, or between an action (clicking a link) and rating (NB. I don't like this, feels like policing in a too visible/restrictive manner that could annoy legit people, while still being faillible if one is ill-intentioned)

[ttencate]

I'm doubtful that tweaking the karma formula would do much. It's too invisible, and even I don't know how it works. A zero-explanation nudge would be better imo.

What if the comment box came first on the page, the ratings below that, and the Save button below both? This would make it quite clear that you're supposed to write something. Quick and dirty mockup:

They're visually grouped together now, and the ordering suggests that ratings follow from comments instead of the other way round. Note how I changed the title from "Game ratings" to "Your feedback" as well.

• What about ratings autosave? It might need to go away.
• What if someone fills it out a second time?
  • Maybe they can't. When they come back after having rated, the comment box is gone and they must use the one below the comment thread.
  • Maybe the box shows their previous comment and acts as an edit box. This is consistent with how ratings work, but could be a bit weird.
  • Otherwise, it should just save the new ratings and append another comment.

---

But because we sort posts oldest-first, you'd need another comment box below the thread. This makes me think... really there are two use cases for the comment box: (1) for players to give feedback on the game, and (2) for developers (and less commonly, other players) to reply to feedback. So how about threaded comments with one level of nesting? It seems to work alright in Facebook, last time I checked. With a Reply button below each comment (and below each reply thread), it would always be clear where your text will be going.

I feel like we've had parts of this discussion before... #151 where I suggested putting comments and ratings into the same form, and #317 where I advocated top-posting for lack of a better idea. Notice how some people do like to read all comments before posting theirs.

[mkalam-alami]

Yes, given the use cases we have for commenting/reading comments, putting the comment box at the top + a reply system would be the most comfortable for everyone. It's true that making people scroll down to comment / look further than the pretty rating box must lose some users.

This approach kinda implies newest-first sorting, which is the part I've had mixed feelings about (newest-first with a box at the top means favoring engagement like social networks, rather than acknowleding discussion then contributing to it). I have yet to find a satisfying way to put comments + ratings at the top plus oldest-first comments.

But thinking about it more:

1. I don't really think most people care about previous comments before posting, they probably just scroll down to submit their own prose ;
2. Nor it is really important: a game submitter getting 3 times the same compliments or criticism but paraphrased is not a bad thing.

A way to sum it up is that we're not dealing with conventional comment threads but more with feedback forms, that may kickstart a discussion (making the single-level reply system especially relevant). So yeah I'm warming up to making the switch :)

So with the comment box now on top we can picture it near the star ratings yes. I like the auto-save, and how it suits tweaking ratings after submitting a comment so I wouldn't mind a cheaper version like:

To handle what happens after saving the first time: we could flag the comment posted this way with a "is_review_comment" flag to make it still editable at the same spot. (Side-effect: this means we need to distiguish the "review comment" form from the box meant for other comments... and that one doesn't necessarily have to be on top?!)

[mkalam-alami]

Getting quited hyped by the "is_review_comment" flag, as it solves another problem: the fact that non-feedback comments (eg. games not working / broken links / other random discussion) are still being counted in the karma formula. Marking reviews as such could let us filter out non-feedback comments.

[ttencate]

How would we make sure that people don't use the "feedback comment" box for such non-feedback comments though?

[mkalam-alami]

I hope having the shiny "your feedback" box with the rating stars and a comment text that stays in place after saving, then a more classic comment box below would be enough hints to let people distinguish their purposes. Worst case the harm wouldn't be more that what we already have currently.

EDIT: Maybe there's UX tips we could take from Gitlab's code review system too, if you're familiar with it.

[ttencate]

FWIW, Discord is split 50/50 about reading other comments before posting your own:

I still think it's an acceptable tradeoff.