Blog
The practical guide to sentiment analysis

The practical guide to sentiment analysis

Real-time understanding of consumer conversations starts with sentiment analysis - opinion mining - that identifies the emotional tone behind consumers’ comments. How do they feel about your brand, product, or service? Love, hate, disdain? Find meaningful data with automated, real-time insights to understand shifts in sentiment, reactions to new products, feelings towards your competitors.

show me the most accurate tool on the market

Introduction

This guide is going to walk you through sentiment analysis. Why you should be using it, what it is, and how to do it. There’ll be real-life brand examples and a demo of the best - and by that I mean, most accurate - tool on the market.

Why is sentiment analysis crucial to your brand? I’ll go into more detail later, but here’s a teaser:

  • Monitor your brand health - understand how consumers feel about your brand. Address pain points, respond to feedback, give them what they want.
  • Manage a potential crisis - spot an issue before it becomes a crisis. Watch for negative spikes and react quickly.
  • Analyze your competitors - monitor conversations about the competition, and steer them towards your brand.
  • Boost performance - track marketing campaigns, product launches, events.

Us humans, we’re great at understanding the tone of a comment. We have contextual understanding and we can recognize praise, anger, indifference, sarcasm. We’re savvy to slang, and we can make sense of abbreviations.

understanding/recognize comment's tone

Do I detect a hint of sarcasm?

Yes. But, would a machine analyzing this tweet, see it as positive or negative? “Great job”, could be viewed as a positive sentiment.

By the power of Grayskull, and a cracking data science team, Talkwalker’s sentiment analysis tool gets sarcasm and would recognize this as a sarcastic and therefore, negative sentiment.

cracking data science

Yeah, great job, guys!

subtle sarcasm

Table of contents

Why is sentiment analysis so important?

Customer feedback - on social media, review sites, forums, your website, etc. - is packed with business insights. You just have to find and understand those insights. Then use them. When you’re listening to what they’re saying, understand how they’re talking about your brand - the sentiment. Get this and you’re on the way to truly knowing your audience. What makes it tick.

Sentiment analysis - opinion mining - will detect a change in public opinion towards your brand, a negative reception to a newly launched product, reactions towards your latest marketing campaigns. For instance, if the sentiment score for a new product is negative, you can research, ask questions, and improve.

Using opinion mining across the board, you’ll find consumer pain points that can be remedied or removed. Consumers love your product, but grouch about slow delivery. Your customer support team has an excellent rep, but your order process is buggy. Monitoring consumers’ attitudes and opinions will reveal areas that should be improved to meet the needs of your audience.

Monitor brand reputation

If you understand the sentiment behind consumers’ comments, you’ll know how they feel towards your brand. Look for changes in sentiment - positive and negative - after implementing a marketing campaign, attending an event, or launching a new product.

Improve customer experience

Tracking sentiment will give your CX team a heads up. They will understand how a consumer feels at every stage of their journey. Any pain points encountered, can be addressed immediately. The customer experience can be more personal, boosting engagement, driving more revenue, and reducing churn.

Managing a crisis

Sentiment analysis will alert you to shifts in opinion. A sudden increase in negative mentions, tackled immediately, can stop an issue becoming a crisis. If you are hit by a PR disaster, measuring the sentiment and filtering the comments by severity of negativity, makes it easier to target your messages. Identifying conversation with positive sentiment means you can amplify those messages - take the edge off.

Competitive benchmarking

Understand how your brand compares in your industry. Monitoring the sentiment surrounding your marketing campaigns and how they affect consumer perception of your brand, in comparison to your competitors, will enable you to tweak what’s not working. And, if you see negative sentiment around a competitor’s campaign, you can steer conversation towards your brand and win a new customer.

Competitive benchmarking

McDonald's vs Burger King - the battle of the sentiment.

In this instance, I’ve compared McDonald’s with Burger King. 13 months of opinion mining, shows McDonald’s has 62% negative sentiment compared with 40.5% for Burger King. Quick Search - Talkwalker’s social media search engine - having analyzed the sentiment, allows you to dig deeper and find the cause of this negativity.

Looking at social media content, blogs, reviews, and new sites for the 13 month period, I found the root cause of the negative sentiment. A YouTube video was posted on October 9, ridiculing the McDonald’s brand.

McDonald's ridiculing video

Quick Search identified the cause of the slump, and gave a negative sentiment score.

YouTube video ridiculing the brand.

"Quick Search provides such an easy and user-friendly opportunity to deep dive into your competitors' social sphere; letting you harness their strengths and weaknesses to improve and cultivate a winning marketing strategy. For a specific breakdown of the importance of this, you should definitely check out Talkwalker's latest article on the necessity and implications of competitor analysis for your business and brand."
Christina Garnett | Marketing Media Maven

Finding brand influencers

Tracking sentiment will not only reveal how consumers feel, but it will help you find influencers that are discussing your brand in a positive way.

If you search for keywords in your industry, you can then find potential influencers that are discussing these topics. Find those that have positive sentiment surrounding their content, and the reaction of their fans and followers, and your industry.

Customer service

Monitoring sentiment is a great way for your customer support team to track consumer dissatisfaction and then address issues before the negativity grows. If your business relies heavily on word-of-mouth marketing - review sites - it’s crucial that you’re in tune with your audience and how they feel about your brand.

If you can turn around these negative comments, appease the consumer and make them happy, you’re demonstrating to other consumers how efficient your brand is. Chances are, you’ll turn that frown upside down and win positive user generated content.

What is sentiment analysis?

Sentiment analysis - opinion mining, emotion AI - is the use of natural language processing (NLP) to analyze online social conversations and determine deeper context - positive, negative, neutral.

And, in English… it’s a way of looking at online chat and working out whether it’s good, bad, or on the fence.

User generated content

Sentiment analysis will reveal a consumer’s opinion with regard to your product, services, team, location, ads, industry, and competitors. It’s a form of social media analytics that can improve your bottom line. The sentiment score of user generated content can steer product development, warn of an escalating crisis, and increase the accuracy of your competitor analysis.

For instance, you’ve launched a new product and you want to know how it’s being received. You could instigate a costly market research survey and spend man-hours making cold calls. Or, you could use opinion mining to understand how everyone feels.

User generated content

Green flag indicates positive sentiment.

Natural language processing (NLP)

Earlier, I casually dropped ‘natural language processing’ into the mix. Also called text analytics, data mining, or computational linguistics, it’s used in social listening to identify and analyze opinions in text. It’s a computer system that processes human language in terms of meaning. It can understand that words can make up a phrase. Phrases can make a sentence. Sentences transmit messages. Following analysis, it marks content as positive, negative, or neutral.

The insights revealed, to be of any use, have to be accurate. But humans aren’t logical, we don’t all speak the same language, we use slang, abbreviations, and acronyms. We express our emotions, and we’re liberal with sarcasm. We change topic mid article. We alter our tone of voice, depending on which channel we’re using. Be it a short tweet, a 2000 word review, or caustic customer feedback. How does a machine grasp the subtle nuances that are so natural to us?

Talkwalker’s sentiment analysis feature understands the human language - 30 languages, to be precise - with up to an average 90% accuracy. It gets the sentiment, it gets the sarcasm. It’ll pull out user generated content and identify the positive and negative sentiment so you understand consumers’ preferences, what they’re passionate about, what they’re unhappy about, so you can improve customer experience, improve product development, and make smart business decisions to improve your bottom line.

Human vs machine

Teaching a machine to be human. Wow! Sarcasm, slang, and irony can still catch humans out - I’m talking to you Alanis. A sarcastic comment. Whoosh, straight over our head. Great, another sarcastic remark I missed. How wonderful. I’ll take a raincheck.

But, on the whole, we get it. For a machine, it has to be translated into objective and quantifiable scores that take into consideration the many nuances that we use when we speak. For instance…

“This guide is totally sick, the author destroyed it. Whoever wrote it is a legend!”

Aw, I’m blushing.

But, an untrained machine would think this is not a good guide, because the author ruined it. In fact, the author doesn’t even exist. The sentiment score of this statement would be negative.

Machine learning enables sentiment analysis to get more and more accurate, as algorithms learn and adapt to the commonalities in conversations and how the context of conversations can change outcomes.

Emotional overload

When we share our opinions on social - comments, reviews, complaints - we also share our emotions. We’re passionate about a new movie, outraged at a lack of customer care, tickled pink that our flight lands on time.

As a brand, you should be analyzing these shared emotions - this sentiment - to understand what matters to consumers. Consumers that you want to love your brand. Use sentiment analysis to monitor online conversations to identify consumers’ emotions. Are they positive, negative, or neutral?

Sentiment analysis - more than one way to skin a cat

Keyword scoring, calculation based on predefined categories, and humans. Three methods adopted by most providers.

Keyword scoring

This method gives words positive or negative scores. Good = positive. Bad = negative. You see the problem?

Hmmm…

The advantage of keyword scoring is that it’s quick, predictable, and doesn’t cost the earth. Rules can be applied but the results are dodgy when used too broadly, because keyword based sentiment analysis doesn’t get context, sarcasm, or slang. Plus, depending on the engineer designating the percentages of what are considered negative and positive words, the results can fluctuate.

Quick and dirty - accuracy levels are roughly between 50 and 80%. Not reliable enough to make smart business decisions.

Predefined categories

This method has users categorizing several results - the training set. Then, an algorithm is employed to decide on future analysis. The level of accuracy is higher than with keyword scoring, but it’s flawed. Man-hours are extensive and the quantity of results are reduced.

While this method is more efficient with regard to the accuracy of the results, you don’t get much for your money. And, to get a handful of results, costs time and money.

Humans

Technology goes out the window with this method. Humans coding sentiment. The accuracy level is sick - hey grandpa, that means it’s good.

Man-hours? Ouch! Trying to catch a comment with negative sentiment and stop it from becoming a crisis, is a race against time.

Cost? Epic fail!

While the results are great, humans interpreting humans is always going to bring the best conclusion, it’s still subjective. Take into account the ever-growing social media world, and there’s way too much content for a team of humans to analyze without the aid of an automation tool.

AI-powered sentiment analysis

Our data science team is so baad. It developed AI-powered sentiment analysis that understands the full meaning behind a sentence - the true meaning. The algorithm will accurately recognize a consumer’s attitude and contextual reactions - in tweets, blogs posts, and articles.

AI powered sentiment analysis

BrandX, are you for real?

Deep learning models are used to simulate the cognitive functions of the human brain, understanding language patterns and basic sarcasm and irony - the algorithm copies how our brains understand.

To achieve a near 90% accuracy rate, our data science gurus had to classify tens of millions of results. Result!

Hasta la vista, baby!

How Merck KGaA uses sentiment analysis to enhance products based on consumer needs

Merck KGaA is one of Germany’s leading science and technology companies. Looking to improve product development, it turned to Talkwalker’s sentiment analysis.

One of Merck’s key products are liquid crystals - those things that manage light in LCD screens. The brand’s dilemma? Should it focus on products that bring a higher resolution and brightness, or products that improve the contrast ratio? What would consumers prefer for their smartphones?

sentiment analysis to enhance products

Comparison of features of high-end smartphones by sentiment.

Results proved that resolution and brightness - along with color and battery - were critical, while consumers didn’t consider contrast to be that important.

Sentiment analysis resolution

Sentiment analysis of resolution.

Digging deeper, it became apparent that the need for high resolution screens has positive sentiment. Armed with insights that answered the question - what do customers want? - Merck focused on enhancing resolution and brightness. Leaving contrast concerns to another day.

In this instance, acute sentiment analysis on a niche topic, brought precise results. Cross-referencing the sentiment data with specific product features, meant that Merck could adapt its product strategy accordingly.

READ THE MERCK KGAA CASE STUDY

The best sentiment analysis tool

Having checked out numerous sentiment analysis tools on the market, I’m not going to waste your time with a great long list. Knowing that your top priority is accuracy, there is only one tool worth mentioning.

Ahem… say hello to Talkwalker.

Sentiment analysis isn’t rated in the social listening industry. Why? Because of all the man-hours involved, which leads to huge expenditure. But our data science team have overcome that, in spectacular fashion.

Talkwalker’s sentiment analysis technology now gives brands the ability to identify consumer sentiment with up to an average 90% accuracy. It’s so sick, I’ll say it again… sentiment analysis with 90% accuracy.

Let’s have a Quick Search look at the last three months of sentiment for Diet Coke.

Sentiment Analysis Tool

Not a great sentiment score, but la di da. Let’s look at January 22, the biggest negative slump. What caused it? Is it an attack on the brand that should be addressed?

Negative slump

Scott Dworkin has 321K followers.

Not an attack on the brand, but worth monitoring. Let’s look at February 6.

worth monitoring

Tony Posnanski has 108K followers.

February 27?

Okay, I’m starting to see a pattern. The brand is not being attacked, but unfortunately, when your product is mentioned in posts poking fun at the President of the US, you’re sentiment score is going to take a bashing.

In January of this year, Tesla had a sudden and unexpected spike in negativity.

Tesla negative spike

Looking closer, the source of negativity became clear.

source of negativity

An article published in Reuters, highlighting a delayed production target - the second delay - of the new Model 3 sedan.

While negativity was to be expected, finding the root-cause quickly meant that Tesla would be able to address the situation head on, before it damaged brand reputation irreparably.

Last year, BlackBerry launched the KEYone. You can see in the graph below, the sentiment surrounding and following the launch.

brand reputation irreparably

Looking good.

Let’s look at the slump on June 5.

brand reputation

Article published in BGR - website about the mobile and consumer electronics market.

The article reveals hardware issues surrounding the new phone, which - if ignored - would damage the brand’s reputation.

Tracking consumer sentiment during a product launch will identify all the positive and negative mentions. In this instance, sentiment analysis has highlighted valuable consumer feedback, meaning the brand can fix the technical flaws and protect its reputation.

The future of sentiment analysis

Exact science? No. But, our data science team absolutely will not stop, ever...

At Talkwalker, we understand that being able to accurately classify sentiment is essential. Brands need to be able to benchmark brand health indicators, supplement the data with demographic information, and combine product features to give consumers what they want.

Our data science team is - as we speak - working to improve the technology even further. 90% today? Tomorrow...

Free eBook: The Definitive Digital Marketing Guide to Getting Results

How do we plan, execute, measure, and analyze marketing campaigns? That’s a real hard question to answer with a single solution. The short answer? Read this eBook. The long answer? Seriously, read this eBook.

My free eBook will help you plan your marketing strategy. How to report on last year, choose your targets, set your KPIs, and monitor the results. It’s packed full of practical and actionable steps, templates, checklists, and real-life examples. There are brainstorming best practices, how to find customer insights that have real value, how to write your vision and mission statements, SWOT analysis and finding SMART goals, identifying your share of voice, and analyzing results.

Learn to listen, listen to learn.

If you’d like to check out Talkwalker's sentiment analysis tool, sign up for a free demo with one of our experts. Seriously, you won't be disappointed!

Free Digital Marketing Guide