Plan

[tmozgach]

• [x] Get data from 2012.1.1 to 2017.12.31;
• [x] Get data about author, karma, data of join;
• [x] Topic Modeling;
• [x] Sentiment analysis;

[tmozgach]

Plan for this week (19-25 Feb):

• [x] Try to train on small data;
• [x] Integrated visualization; https://github.com/shichaoji/easyLDA/blob/master/build/lib/easyLDA/base.py
• [x] Understand parameters; https://markroxor.github.io/gensim/static/notebooks/lda_training_tips.html
• [x] Introduction to Cedar;

[tmozgach]

2.0 Text analysis on the title and the body of the questions in the thread.

1) for the titles

• [x] a) conduct topic modelling trying 30, 40, 50 as topic numbers
• [x] b) check length of the words. (It’s the number of words)
• [x] d) check the sentiment being positive or negative

2) for the bodies

• [x] a) conduct topic modelling trying 30, 40, 50 as topic numbers
• [x] b) check length of the words
• [x] c) check the sentiment being positive or negative

[tmozgach]

• [x] Delete first row (col1)
• [x] Check Topic Modeling parameters;
• [x] Run on Cedar;
• [x] Combine everything together;
• [x] Write a guide;
• [x] Submit everything;

[tmozgach]

• [ ] check if it is a question format (e.g., starting with ‘what', ‘how’ or end with question mark), or statement format https://github.com/tmozgach/ent_ob/issues/11