search

mozilla / addons

☂ Umbrella repository for Mozilla Addons ✨
Other
125 stars 41 forks source link

Finding solutions for toxic reviews on Firefox Add-ons #7396

Closed dessant closed 4 years ago

dessant commented 4 years ago

Firefox Add-ons has an issue of toxic reviews, which exposes developers to unnecessary stress, and makes them less likely to read and respond to reviews, out of precaution for their own mental well-being.

This toxicity not just affects developers, but projects suffer too, because developers are less likely to try finding solutions to issues in a toxic review, even if they report a valid issue. Moderators are also exposed to this toxicity when the content is reported.

There are automated solutions to filter out some of the toxic reviews, such as https://www.perspectiveapi.com.

Consider this review that is marked by Perspective as likely toxic:

funktioniert nicht,Dreck Müll

Nobody gains anything from reading such reviews. Reviews with usable content are also likely not worth saving if they contain toxic language, because whatever information they contain is probably not worth the stress they propagate to developers and moderators, and they would be eventually removed anyway, once flagged.

A possible solution would be to check future reviews for toxicity and automatically remove them if they are found to be toxic, while also notifying users that their review has been moderated. The rating could be kept and only the review removed, if you are worried about the loss off that signal.

This will hopefully lead to some users understanding that abusiveness is not tolerated on this platform, and offer them a chance to rephrase their message in a less toxic way.

eviljeff commented 4 years ago

@jvillalobos to opine. Though I'll note we had Akismet integration for anti-spam and ended up removing it again as it wasn't useful enough in reality.

jvillalobos commented 4 years ago

Yes, like Andrew mentioned, we tried to implement something with Akismet, but targeting spam rather than toxicity. I think the learnings there apply to this as well: it takes a lot of human time investment to train a system to align to our expectations, and even with that investment it's not clear it will ever reach a satisfactory state.

I agree we haven't spent enough time working on review moderation. We put some ideas on hold, like meta-moderation, which we should revisit eventually. We might also enable reviewing from the Add-ons Manager in the future, which will introduce new challenges. It's just not something we have the bandwidth to focus on right now.

sbs2001 commented 4 years ago

What's the status of this issue ? I would like to help.

I think this can be solved by using a ML model for a kind of sentiment analysis. There are tons of freely available datasets of comments with their intent, https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/data is a good place to start .

I haven't dug deep enough, but I think there might be some open source models specifically designed for this purpose , all we have do is fit and integrate that model on the FIrefox add on server.

Let me know your opinions and approaches , I will start my research and probably build some prototypes.

jvillalobos commented 4 years ago

Thank you for the suggestions. Unfortunately I don't think we have the resources to set this up and keep it running. We need people to be able to verify that this system is working reliably, and make corrections when needed. We currently don't have that.

I'm closing this issue for now, but I hope we can revisit these possibilities later in the year.

dessant commented 4 years ago

@jvillalobos, this issue was opened to start a discussion about possible solutions to toxic reviews. By closing the issue and reducing its visibility, the discussion we were hoping to initiate is effectively shut down.

The fact that it received a community response so quickly shows that there is interest in sharing ideas regarding this issue, and I think it would be unfortunate to bury the discussion.

eviljeff commented 4 years ago

This issue tracker isn't a discussion forum - see https://discourse.mozilla.org/c/add-ons/35 for that. Obviously any possible solution is going to run up against the same constraints though of a) engineering capacity to implement the solution and b) operational capacity to train/manage the solution once implemented.