NLP - description of the assignments and projects in reverse chronological order

Project

Outline
Documentation

Project structure

Root directory: Project/Emocontext

Setting up the environment (tutorial) - for running the notebooks locally
1. Install virtualenv: python -m pip install virtualenv
2. cd Project
3. Activate the virtualenv: Project\Emocontext\Scripts\activate.bat (Windows) OR source Emocontext/bin/activate (Linux)
4. Install
jupyter notebook: python -m install jupyter notebook
requirements: python -m pip install -r Project\Emocontext\requirements.txt
1. Launch jupyter: jupyter notebook
  (To deactivate the virtual environment use: deacivate)
Jupyter notebook cheatsheet:
- jupyter notebook --generate-config
  Kaggle notebooks can be uploaded on kaggle and run in cloud

Python files (extension .py) are run with Python 3.6.3: _python .py_.

twitter_tokenizer.py (a.-g.)
mwe.py (h.)
Data and results files
microblog2011_tokenized.txt - tokenized corpus
tokens.txt - list of all tokens and their frequencies
Aditional files:
data/cleaned_twitter_training_corpus.txt - corpus merged from 2 sources (SemEval task 2013 and Kaggle dataset of customer support twitter data) used for training the model for the multiword expression acquisition
data/list_of_stopwords.txt
data/list_of_contractions.txt - list of contracted words for which nltk's tokenizer splits the words, unlike the twitter tokenizer
External resources

Python libraries:
nltk - tokenizer and wordnet
gensim.models.phrases
Part 2 - Mozhgan Nasr Azadani

Files

Java files can be added as an projects in a favourite java IDE. No external library is needed. Moreover, On mac systems, they can be run directly as ./.java in the correct directory using terminal.

Code files
format-change.java (a.)
accurracy.zip (b.)
Frequency.zip (c.)
Data and results files
POS_results.txt - POS tagger output for the whole corpus
External resources

CMU POS tagger