This script reformats the lens data pulled in using their API to be fit MATCH's data standards. This script also pulls labels from our labeled csv data to label this Lens API dataset.
Related Issues
Resolves Issue #43
Backwards incompatibilities
None.
New Dependencies
Imports unique when compared to other files are:
re - (RegEx for removing/replacing punctuation)
nltk - (NLTK for intelligently tokenizing words and providing the corpus of stopwords)
ast - (AST for safely parsing stringified lists)
string - (For providing the full set of special characters)
Summary
This script reformats the lens data pulled in using their API to be fit MATCH's data standards. This script also pulls labels from our labeled csv data to label this Lens API dataset.
Related Issues
Backwards incompatibilities
None.
New Dependencies
Imports unique when compared to other files are: re - (RegEx for removing/replacing punctuation) nltk - (NLTK for intelligently tokenizing words and providing the corpus of stopwords) ast - (AST for safely parsing stringified lists) string - (For providing the full set of special characters)