search

dwyl / comments-you-will-regret-blocker-chrome-extension

:speak_no_evil: [WiP] A Chrome Extension that monitors comments typed into a textarea and blocks you from posting silly things you will regret!
GNU General Public License v2.0
9 stars 1 forks source link

Write parsing engine from scratch or use something that's already out there? #6

Open ryanpcmcquen opened 7 years ago

ryanpcmcquen commented 7 years ago

Is anyone aware of an engine that analyzes language in the way this project wants? It'd be fun to work on it ourselves but it will be quite an undertaking. 🦄

nelsonic commented 7 years ago

@ryanpcmcquen writing a Natural Language / Machine Learning engine is way beyond the scope of this Chrome Extension project. We need to leverage the existing Open Source projects which have millions of hours invested in them. ⌛️ 👍

@perborgen / @nikhilaravi / @jrans might point us in a fruitful direction on this. ❤️ ✅ We need to use the best-in-class NL/ML packages regardless of the language they are written. This might be a place to start: https://opensource.com/business/15/7/five-open-source-nlp-tools

Heck, if @Palantir has something Open Source we can use, that will solve the "backend" of this project I'm happy to give them access to all my keystrokes just to avoid posting offensive comments ever again! 😧

perborgen commented 7 years ago

I've only got experience with Python, Scikit Learn and the NLTK package in regards to this. But those tools should be able to do the job.

I'd guess it's fairly easy to get a classifier which classifies clearly offensive comments, but subtle isms would be harder to catch. Best approach is probably to find a trained network/algorithm to use.

Seen this one? http://fortune.com/2016/07/30/new-ai-spots-online-abuse/

nikhilaravi commented 7 years ago

@perborgen where do you think we could get training data for the classifier? A simple idea would be to manually create a training set of messages with content we think is offensive/inoffensive (i.e. labelled 1 or 0 with offensive/non offensive) and train a naive bayes classifier http://scikit-learn.org/stable/modules/naive_bayes.html. This would mainly be based on the occurence of words and not really take into account the word ordering, emojis punctuation, capitalizing etc..

Another ideas is to use content embeddings like paragraph2vec (this paper has an example of using paragraph2vec for hate speech detection in yahoo comments) but again it would require a labelled training set to train a binary classifier.