hypothesis / product-backlog

Where new feature ideas and current bugs for the Hypothesis product live
118 stars 7 forks source link

AI based moderation for toxicity, spam and AI detection #1453

Open leedenison opened 1 year ago

nairiboo commented 1 year ago

Adding to this; the current method of moderation has a large impact on support resources as it is currently a manual process and not scalable.

mkdir-washington-edu commented 1 year ago

I don't know what happened to the description here, but I think this issue came from this thread: https://hypothes-is.slack.com/archives/C2BLQDKHA/p1686943086486359?thread_ts=1686934813.780349&cid=C2BLQDKHA

The web app has a number of accounts that are likely created by spammers. While we can manually detect spammers when they post on our site, or when their annotations get flagged, we don't have another way to detect them. Once detected we can NIPSA them (or delete their accounts, but we've never done that as far as I know), but we don't have other tools.

The support team investigated the last time we saw a huge number of new accounts made in a short time and we saw some trends in Spam accounts on our site:

Some thoughts: