sim353 distance check

[JherezTaylor]

Get the distance of the word pairs in the wordsim353 dataset using WordNet. Check if the distance matches up with the variation of the individual annotator scores in sim353 in an effort to see if we can measure similarity or dissimilarity in that what. This should inform if any domain is being reflected as well.

An initial pass of sim353 shows that most of the variation in the scores come down to subjective opinion on things like:

• Vocabulary error
• Object vs idea
• Rating vs topical similarity
• Person vs idea
• Place vs activity
• Object vs activity
• Idea vs idea

[JherezTaylor]

Look into this - https://github.com/alvations/pywsd/blob/master/pywsd/similarity.py#L76

[JherezTaylor]

http://www.nltk.org/howto/wordnet.html http://stackoverflow.com/questions/22031968/how-to-find-distance-between-two-synset-using-python-nltk-in-wordnet-hierarchy

[JherezTaylor]

Closing, I don't think I'll use this anymore.