Change ratings to 0-10 scale

[dfabulich]

The current rating system uses a 1-5 star scale, but we could use a 0-10 scale instead.

[qdacsvx]

If all the game ratings are currently 1-5 how would they be converted to 0-10? Could users perform the translation? What if I want half my 3s to be 6s but half to be 5s - only the user knows which should be 5s and which 6s.

Users are used to rating games out of 5. Will it be forbidden to rate games out of 5? What if users publish ratings out of 5 e.g. in review text?

0/10 ratings? Suppose 1000 game ratings are changed to 0/10 on the day IFDB.ORG goes live. Good / bad / ok? Should moderators do something or is it fine?

This seems like two proposals:

1. use ratings out of 10 for game ratings.
2. add a 0 rating.

[huftis]

I agree that a more fined-grained rating system would be an improvement, but I think a 1–10 (instead of a 0–10) rating system would be better. Then the ratings will be compatible with the IFComp ratings. This makes it easy for the users to submit their IFComp ratings to the IFDB (perhaps there could even be an option of doing this automatically?).

One possible implementation (and way of displaying the ratings) would be to add support for giving ‘half stars’ (e.g., ★★★⯪☆). This would give 10 possible ratings (from ⯪☆☆☆☆ to ★★★★★, i.e., ½ to 5). To convert to or from the IFComp rating system, you only have to multiply or divide by two. So a 5-star system with half stars is fully compatible with a numeric (or whole-star based) 1–10 rating system. (Personally, I think it’s easier to choose a rating on a 5-star system, even though they’re theoretically equivalent.)

To convert the old ratings, one could either keep using them directly (accepting that a 1/5 is no longer the minimum value), or, preferably, offer the user the option to easily manually convert them. This can be done by having a page with a simple table:

Game title | Old rating | New rating | (‘Use suggested rating’ button)

For each game, the user could then select the new rating (which would be pre-filled with the old rating). If they’re happy with the pre-filled option, they can click the ‘Use suggest rating’ button.

The database would need to store a flag for each rating which tells us if the rating is of the old style or new style. Only old-style ratings would be shown on the ‘update your old ratings’ page, and the user wouldn’t have to update all their old-style ratings in one go. If they don’t update a rating, the old rating would be used.

[qdacsvx]

+1 I agree with much in your post @huftis. Personally, I'm fond of ratings out of 100! But it takes longer and longer to rate things the more subtle the rating system so it might not be practical for every day use. I think people find ratings out of 10 (i.e. 1-10) intuitive - more than ratings out of 5.

I think 1/2 s might be quite fiddly. But if IFDB's s were 1cm+ square they would be easy to hit. Anyways, I use the accessibility voting option which is a popup menu. Like IF Comp voting it will be quite large with 10 options. Depending where the screen is scrolled it pops up at the top or the bottom so I constantly have to move the mouse around.

A drawback of continuing to use low resolution ratings in a high resolution system is that voters/IFDB wouldn't benefit from the higher resolution available. There should be a push to upgrade your ratings. Perhaps a: a few frequent fiction points for upgrading one page of games. b: make frequent fiction point changes visible iftechfoundation/ifdb#805 . c: a list of unupgraded votes on your homepage.

A way to square the circle of users want 3-s to become 5-s and 6-*s just by clicking "convert" but IFDB doesn't know which 3s should become 5s and which 6s: If users can rank their voted-for games (or just the 3s) now IFDB will know which are the "high" 3s which should become 6's and which the "low" 3s which should become 5s.

I like that users can go on rating out of 5 until they get comfortable to change (if ever). If nobody likes ratings out of 10 would it be easy to go back to 1-5 ratings?

Zombie users won't be able to update their ratings.

Regarding 0 votes. What * representation?

• --------- 1 ---------- 0 ---------- no vote (the same as 0*)

will it be ambiguous to use the same representation as "no vote"?

In researching rating systems I noticed a plausible phenomenon where users spontaneously vote for a max (or min) rating in consensus and attribute special value to the "pure" max (min) score which results. This suggests there could be more 0s than predicted if users pile into consensus seeing a "pure" 0. As for impact on the authors of 0-rated games. I would feel it worse if it was by people I know than if it was by one-time logins.

[huftis]

+1 I agree with much in your post @huftis. Personally, I'm fond of ratings out of 100! But it takes longer and longer to rate things the more subtle the rating system so it might not be practical for every day use. I think people find ratings out of 10 (i.e. 1-10) intuitive - more than ratings out of 5.

The way I like to think of 1–5 ratings with half-stars, is that you can choose to use only whole stars if you prefer. But if you think that an entry isn’t quite good enough to get a 5, you can instead give it a 4½. Or if you think that an entry is somewhere between a 3 and 4, you can give at a 3½. So the extra granularity is there if want it, but you can also choose to ignore it (always or some of the time).

That said, I think a 1–10 rating is also quite acceptable. With this rating system, there are so many stars that it’s a bit difficult to count them. So if this is implemented, the numeric rating should (also) be shown.

(It’s of course also possible to add a user preference for showing the results as 1–10 or 1–5. But I’m wary of adding too many preferences.)

I think 1/2 s might be quite fiddly. But if IFDB's s were 1cm+ square they would be easy to hit.

I think current IFDB stars are already quite fiddly to hit, so they should be made bigger in any case.

One way of handling the half star selection that I’ve seen a few places, is that the first click on star gives a whole-star rating, but if you click it a second time, the star turns into a half star. I think that works pretty well (but it’s perhaps not so easy to discover this feature).

Anyways, I use the accessibility voting option which is a popup menu. Like IF Comp voting it will be quite large with 10 options. Depending where the screen is scrolled it pops up at the top or the bottom so I constantly have to move the mouse around.

I like the way the IMDB handles this. There’s a ‘Rate This’ button/area:

If you click it, a star-rating widget pop-ups, where you can select a star, and the ‘Rate This’ text turns into your currently selected rating:

This uses JavaScript, so there would of course need to be a non-JavaScript accessibility backup.

A drawback of continuing to use low resolution ratings in a high resolution system is that voters/IFDB wouldn't benefit from the higher resolution available. There should be a push to upgrade your ratings. Perhaps a: a few frequent fiction points for upgrading one page of games. b: make frequent fiction point changes visible iftechfoundation/ifdb#805 . c: a list of unupgraded votes on your homepage.

Yes, something like that sounds like a good idea.

A way to square the circle of users want 3-s to become 5-s and 6-*s just by clicking "convert" but IFDB doesn't know which 3s should become 5s and which 6s: If users can rank their voted-for games (or just the 3s) now IFDB will know which are the "high" 3s which should become 6's and which the "low" 3s which should become 5s.

That’s almost how I rate the IFComp games. :) First I often give ‘preliminary’ ratings. Then if I see that have given very many games a 7, I rank them, and see if some the lowest-ranking ones don’t really deserve a 7, and can be pushed down to a 6, or if some of the highest-raking ones could be pushed up to an 8. But I’m not sure an automatic scoring algorithm based on ranking would be good enough.

Zombie users won't be able to update their ratings.

No. And I don’t think that’s a big problem.

Regarding 0 votes. What * representation?

• --------- 1 ---------- 0 ---------- no vote (the same as 0*)

will it be ambiguous to use the same representation as "no vote"?

I vote for a 1–5 or 1–10 rating system where there is no 0 rating (it’s often confusing if a 0 rating is the lowest possible rating or a “no rating”), but with an “no rating”/“remove rating” option. I like how this is implemend at the IMDB. There is an explicit “x” (on the left) to remove the rating:

[qdacsvx]

In the terms of the voting gadget, I would prefer a 1-10 rating. The IMDB design with stars lighting up following the mouse cursor looks practical for me.

Converting 1-5 ratings to 1-10 ratings.

I think the ratings should be sorted so all the 3s etc are together. Also chronologically. There could be two target columns e.g. for old rating 3 there could be a target column for 5 and a target column for 6. You would just need to tick one choice for each rating. The higher one could be ticked by default so you only need to change roughly half. For a more compact mobile phone layout, the rating boxes could be in a bar underneath the game.

For games I played years ago, unfortunately, I can't remember how I ranked same-rated games. It would be helpful to show a search-style listing not just the title so it is easier to identify the game.

I don't know a simple way to submit game rankings to IFDB. Maybe you could type the rank you want into a text box next to each game. Not faster than choosing the "high" 3 ratings.

I think the subtext of imagining submitting rankings to ifdb is that I used to prefer ranking games and only reluctantly took up rating. But I found ranking something into more than 30 is slower than rating. So I moved to rating. A frustration of rating is that many works I rank differently end up with the same rating.

Advantages of 0?

Adding a new rating allows voters to express a new distinction, between 0 and 1 which voters could not in a 1-10 scale.

Some voters may have refrained from rating very poor works on the ground that they should be rated 0, but there was no 0 in the allowed ratings. If voters are liberated, IFDB could gain additional votes.

Voters who wanted to rate something 0 but rated it 1 instead could rate it more accurately.

[brirush84]

In education, they've done research to show that adding more levels of grades tends to stress students out and cause more problems. For instance, having ABCDF is better than A A- B+..., because getting a B is clearly worse than getting an A, but getting an A- makes you feel like you were so close to an A...

I don't believe going to a 10-pt rating scale would significantly improve the IFDB experience.

[alice-blue]

If you change the scale, it would be harder to accurately compare old ratings (that don't get updated) with new ratings, which might make e.g. the top 100 less accurate (especially when comparing old games, which will have a bunch of old ratings, with new games).

If you change the scale to 10 stars, and auto-translate people's old ratings, that would look misleading. Seeing a 6/10 star rating next to someone's old review implies that they actually gave the game 6 stars, which they didn't. It could be off in either direction.

Changing to a system that allows half stars, but is still out of 5, would not appear as misleading when translating people's old ratings, but you'd still have a potential accuracy issue when comparing games with more old votes vs. games with more new votes.