Expand POMS keywords - Githubissues

vlandeiro / sporty-twitters

Project trying to correlate the practice of sports of Twitter users with their well-being.

0 stars 1 forks source link

Expand POMS keywords #12

Closed aronwc closed 10 years ago

aronwc commented 10 years ago

We discussed some options:

WordNet
Search for words that appear in similar contexts

vlandeiro commented 10 years ago

I have started to try expanding the words by finding the most frequent co-ocurrences for each word from the original list. However, the results are not convincing as a lot of words are amongst the most frequent ones for each poms word.

aronwc commented 10 years ago

Ideas to try:

use set instead of list (for efficiency)
remove punctuation
split POMS words by category

vlandeiro commented 10 years ago

Categorized POMS words: http://www2.ul.ie/pdf/526781607.pdf

vlandeiro commented 10 years ago

I used WordNet to find synonyms to the POMS words and I am currently trying to develop an algorithm to find words that have the most similar context when compared to the POMS word contexts.

aronwc commented 10 years ago

wordnet synonyms expanded from ~60 to ~350 words.
fix the "on" word in TA
remove mentions
create context dicts like: happy: [to@-1=100, be@1=30, ...]
compute cosine similarity between each POMS word and all other words
report most similar words.

aronwc commented 10 years ago

also consider Yahoo data

vlandeiro commented 10 years ago

"on" word fixed in TA from file poms_extended
building context dicts lead to a performance issue when processing more than 300K tweets
implemented cosine similarity computation

aronwc commented 10 years ago

try restricting to at-3, at-2, at-1, at1, at2, at3

aronwc commented 10 years ago

Latest results look pretty good -- top 50 words for anxiety/depression seem reasonable.

Next steps: labeling some tweets (~1000) that contain at least one of the keywords. Binary label: does the tweet express something about the emotional state of the user or not? (e.g., filter out movie quotes).