Natural language processing project based on the one-million-posts dataset.
More than 3.000 user comments are written each day on www.derstandard.at. Moderators need to review these comments regarding several aspects like inappropriate language, discriminating content, off-topic comments, questions that need to be answered, and more.
We provide machine learning models that detect potentially problematic comments to ease the moderators' daily work.
pyenv install 3.8.5
make setup
. For further instructions on how to run our code see SETUP.md.
The presentations are found in ./presentations/ |
Presentation file | Description |
---|---|---|
OneMillionPosts-GraduationEvent.pdf | Presentation of the graduation event from April 28, 2021 | |
OneMillionPosts-Midterm.pdf | Midterm presentation of the project from April 12, 2021 | |
OneMillionPosts-AnnotationComposition.pdf | EDA concerning ticket #24, #25 |
The models' code is found in ./modeling/ in this repo.
They are pushed as .py files. See SETUP.md. |
Model | Description |
---|---|---|
gbert Classifier | German BERT base | |
Zero Shot Classifier | xlm-roberta-large-xnli | |
XGBoost | XGBoost | |
Logistic Regression | Logistic Regresssion | |
Support Vector Classifier | Support Vector Classifier | |
Random Forest Classifier | Random Forest Classifier | |
Naive Bayes Classifier | Naive Bayes Classifier | |
LightGBM | LightGBM algorithm not considered for further modeling |
The notebooks are found in ./notebooks/
.
They are pushed as .py
files. See SETUP.md.