Finding and removing Stopwords - Githubissues

TUM-IDP-WS-20 / doc

0 stars 0 forks source link

Finding and removing Stopwords #18

Closed farukcankaya closed 3 years ago

farukcankaya commented 3 years ago

Parent: #2

Find stopwords on the accounting domain:

[x] Use generic stopwords in https://sraf.nd.edu/textual-analysis/resources/ that are published regularly. (Florian indicated that it might be %50 applicable)
[ ] Calculate Word frequency.
- [ ] option1: by year
- [ ] option2: all dataset
- [ ] option3: by journal

farukcankaya commented 3 years ago

Found stopwords: https://drive.google.com/drive/u/0/folders/1lHVGwOFRm_rqMA0RzAoqlzr-YcnFsSn7

University of Notre Dame StopWords

8 => StopWords_Auditor.txt
190 => StopWords_Currencies.txt
116 => StopWords_DatesandNumbers.txt
121 => StopWords_Generic.txt
570 => StopWords_GenericLong.txt
199 => StopWords_Geographic.txt
11905 => StopWords_Names.txt

MALLET STOPWORDS:

523 => English
125 => French

and created a website from a model that uses stopwords above: https://simple-recommender.herokuapp.com/ You can find the repo of the website here: https://github.com/TUM-IDP-WS-20/simple-recommender