search

clips / pattern

Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
https://github.com/clips/pattern/wiki
BSD 3-Clause "New" or "Revised" License
8.75k stars 1.58k forks source link

Financial data sentimental analysis return low polarity #145

Closed ghost closed 7 years ago

ghost commented 8 years ago

I use polarity function to asses the sentiment of financial data; what I observed is pattern polarity function tends to give false negatives in a financial data.

A article contains this phrase tends to give negative result " .....industry is going up and stop loss should be placed at 20..."

I think pattern mis interprets stop loss as a negative meaning.

There are many financial sentiment dictionaries available in web can we use those dictionaries with pattern. If yes how can we do it?

jburb commented 8 years ago

IIRC the sentiment analyzer object has an "annotate" method that allows you to assign polarity and subjectivity float values to a word. I think you could just use that iteratively.

If you want I can post an example.

ghost commented 8 years ago

An example post would be nice. It would be helpful to all people who currently face similar problem

polarity and subjectivity float values to a word do "stop loss" will be considered as a word

jburb commented 8 years ago

Unfortunately that may be a wrinkle in the annotation approach. I can't recall for sure though. Will post what I know later today.

jburb commented 8 years ago

Well here is the snippet from sentiment class in /text/init.py.. Sorry for any false hope I may have given. Perhaps someone who knows more can give a better answer.

def annotate(self, word, pos=None, polarity=0.0, subjectivity=0.0, intensity=1.0, label=None): """ Annotates the given word with polarity, subjectivity and intensity scores, and optionally a semantic label (e.g., MOOD for emoticons, IRONY for "(!)"). """ w = self.setdefault(word, {}) w[pos] = w[None] = (polarity, subjectivity, intensity) if label: self.labeler[word] = label

Anyway if you were to use above method it could be something like:

foo_analyzer = pattern.sentiment foo_analyzer.annotate('unsetword', pos='NN', polarity=-1.0, subjectivity=0.7, intensity=1.0)

You could always try a second or alternate pass of sentiment analysis with a classifier E.G. from TextBlob which is supposed to be Pattern-compatible. Again sorry that annotate may not be your answer.