NLP: Article Sentiment Analysis

[joelansbro]

As other NLP jobs to test out, a sentiment analysis of the blog article would be useful to gain a prior insight into the possible direction of an article. Sentiment analysis is a widely studied and implemented NLP workload, and would be useful to include. (Say you could use the keyword tracking over-time, and also include sentiment towards an article over time!)

So also include this in a NLP investigation (also find some scholar articles to cite with their prior work in sentiment analysis over grey literature, if possible)

[joelansbro]

Created a prototype testing out sentiment analysis, the next step will be to run a few of the articles over the sentiment analysis in order to get a feel for how good the sentiment analysis has gone, Get a feel for how well the sentiment analysis scores have captured a blog tone, and present the numbers. ending with a summary of findings (can be rough notes) that will be able to lead into explaining in the project.

[joelansbro]

Added TextBlob over about 40 articles, and took down a look into the relationship between polarity and word count, as well as the range between the lowest and highest polarity, to see if this is low.

[joelansbro]

TODO:

• Compare VADER and Flair to see if these are better, in terms of polarity range / word_count over polarity. - this is called Ensemble Learning
• Increase the count of articles to 100
• Read the Sawalha article on Sentiment Analysis for inspiration

[joelansbro]

I've got some good notes on the investigation within the PR now, including graph metrics against data, which should be adequate for writing up and justifying inclusion in the pipeline,

there are some downsides to the sentiment gained - negative articles don't seem to be captured very well for instance, this could be done more accurately for instance with Flair, or some kind of change to how we parse through and aggregate sentences into VADER's compound scoring.