itchio / itch.io

:bug: Public itch.io issues tracker and documentation - use support instead for private information!
https://itch.io/support
241 stars 27 forks source link

Add ratings data export for jam hosts #1622

Open hwashere opened 4 months ago

hwashere commented 4 months ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

For Jams where submissions are voted on by submitters, it is common to see 'review bombing' (unnecessarily unfair reviews).

Describe the solution you'd like A clear and concise description of what you want to happen.

The ability for the host of jams to be able to export raw ratings data when the rating window is over to better moderate the scores.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

I have contacted support (who have given me the data) but this took 7 days+ after the rating window had finished.

Additional context Add any other context or screenshots about the feature request here.

leafo commented 4 months ago

Are you looking for individual ratings from participants? Typically for the security of those who participate in jams, their direct votes are not made available, but in some circumstances we may be able to provide jam hosts with that information if there are integrity issues.

hwashere commented 4 months ago

Yes I am looking for the individual ratings data for the purpose of score moderation - but if security is a concern (rightly), then it would be totally fine to anonymise the ratings data.

Our jams are community voted and often have cash prizes, so it would be very helpful for us to be able to moderate the scores & remove obvious review 'bombs' (i.e. 0/1 stars across all criteria). In one we hosted earlier this year ~25-30% of ratings were like this; this feedback was provided by entrants.

We don't need to know who gave the rating, just the ability to moderate the final scores and remove any obviously unfair ones, so the final standings are as fair as possible.

Thanks for the response!

leafo commented 4 months ago

@hwashere Have you written in a support ticket for this particular issue? If so can you send one in and then send me the ticket ID that comes in the receipt email. I want to review the jam in question, and you can share info about your jam in there as well about why you feel there are review bombers. I can also provide you with instructions on accessing an export.

https://itch.io/support

hwashere commented 3 months ago

@leafo sorry for the delayed reply, I have not written a support ticket directly for this - but I raised a ticket to ask for an export and this was fulfilled (support ticket #213743). However this provided the same data I can access through the UI (i.e. top level aggregate scores) - to be clear I would love to be able to moderate obviously unfair individual ratings on submissions?

This was feedback left in our discord:

image

Another left a message saying 35% of his reviews were similar.

Both of these were participants in our jam: https://itch.io/jam/gamemaker-game-jam-2024