How we do Sentiment Analysis and Text Analytics with ML

Recently, Sentiment Analysis and Text Analytics have become buzzwords and rightly so.

In the world of market research,

Open-ended text is one of the richest sources of customer feedback.

And, asking your customers for direct feedback is a simple yet effective way to get actionable information that can drive business decisions.

However, there is a catch.

It is impossible to manually go through exhaustive feedback or social media comments.  

This is where Sentiment and Text Analytics come in

Recently, Sentiment Analysis and Text Analytics have become buzzwords and rightly so.

There is a huge wealth of data in customer feedback surveys that can be mined using Machine Learning Algorithms. Now a days, it’s possible to train machines to read and analyse text across millions of comments, quickly and accurately.

Therefore, this article explains how we use Sentiment and Text Analytics to analyse open ended comments.

HOW WE DO SENTIMENT AND TEXT ANALYTICS

Renowned analyst, Meta S. Brown, defines Text Analytics as converting text into conventional data.

It refers to the process of deriving important information from ‘unstructured’ written text.

In the context of business and marketing, it usually means analysing a continuous stream of free text (feedbacks and comments) either written by or about the customers.

Once a continuous data stream of customer feedback has been set up, the process of Text Analytics can then be implemented. There are essentially two things that you need to identify via Text Mining-

1. Sentiment Analysis– You try to understand your customer’s overall sentiments behind a comment or feedback. First, you try to decode whether the comment is positive or negative. This involves understanding the quality of the comment, in binary terms of Yes (for positive) and No (for negative).
   
   
2. Cause– For instance- If a comment is positive, why is it so? This step includes figuring out what the customer is saying within the comment that classifies it as a positive or a negative statement.

Numr takes this one step further with Predictive Analytics

PREDICTIVE TEXT ANALYTICS

For Predictive Text Analytics, we first need to devise an algorithm that can scan through the textual data. This is done using Machine Learning. Numr uses an SVM (Support Vector Machine) method for this process. What it mainly does is, look at a cluster of words and try to understand which category to group them under.

But first, we need to manually categorise around a thousand comments for it to work accurately. The reason being that the Machine algorithm needs a model (based on a human perspective of classification) to base its calculations on.

Once you have categorised around a thousand comments, there are primarily two steps in the process-

STEP 1– About 70% of the categorised data is used as a Training Set. The machine then, builds a Model based on this set.

STEP 2– The remaining 30% of pre-categorised data is used as a Test Set, to check if the Model is able to predict the results accurately.

This process makes available two important pieces of information-

1. The percentage of comments that the Predictive Model was able to classify
2. The accuracy of that classification.

If required, the model can then be tweaked accordingly.

Everyone now knows that 80 percent of business-relevant information originates in unstructured form, primarily text.

Knowing how to interpret data via Text Analytics enables marketers to measure, manage and analyze marketing performance to maximize its effectiveness and optimize profits.