Implement Data Cleaning

melisMirza / Covitter

Repository of SWE 573 project

0 stars 0 forks source link

Implement Data Cleaning #18

Closed melisMirza closed 3 years ago

melisMirza commented 3 years ago

Write the code to implement the data cleaning strategy.

Have the methods modularized in a separate class that can be imported.
Keep the custom stopwords and abbreviations on different files and read them into the code.

melisMirza commented 3 years ago

Cleaner.py is created and it will be the main module for cleaning. Each step is lemmatization, removal of stop word etc. are implemented as different functions to provide flexibility.