The practical guide to sentiment analysis
Real-time understanding of consumer conversations starts with sentiment analysis - opinion mining - that identifies the emotional tone behind users’ comments. How do they feel about your brand, product, or service? Love, hate, disdain? Find data with automated, real-time insights to understand shifts in sentiment, reactions to new products, feelings towards your competitors.
This guide is going to walk you through sentiment analysis. Why you should be using it, what it is, and how to do it. There’ll be real-life brand examples and a demo of the best - and by that I mean, most accurate - tool on the market.
Why is sentiment analysis crucial to your brand? I’ll go into more detail later, but here’s a teaser:
- Monitor your brand health - understand how consumers feel about your brand. Address pain points, respond to feedback, give them what they want.
- Manage a potential crisis - spot an issue before it becomes a crisis. Watch for negative spikes and react quickly.
- Analyze your competitors - monitor conversations about the competition, and steer them towards your brand.
- Boost performance - track marketing campaigns, product launches, events.
Us humans, we’re great at understanding the tone of a comment. We have contextual understanding and we can recognize praise, anger, indifference, sarcasm. We’re savvy to slang, and we can make sense of abbreviations.
Do I detect a hint of sarcasm?
Yes. But, would a machine analyzing this tweet, see it as positive or negative? “Great job”, could be viewed as a positive sentiment.
By the power of Grayskull, and a cracking data science team, Talkwalker’s sentiment analysis tool gets sarcasm and would recognize this as a sarcastic and therefore, negative sentiment.
Yeah, great job, guys!
Table of contents
- Why is sentiment analysis so important?
- What is sentiment analysis?
- Sentiment analysis - more than one way to skin a cat
- How Merck KGaA uses sentiment analysis to enhance products based on consumer needs
- The best sentiment analysis tool
- Free eBook: The Definitive Digital Marketing Guide to Getting Results
Customer feedback - on social media, review sites, forums, your website, etc. - is packed with business insights. You just have to find and understand those insights. Then use them. When you’re listening to what they’re saying, understand how they’re talking about your brand - the sentiment. Get this and you’re on the way to truly knowing your audience. What makes it tick.
Sentiment analysis - opinion mining - will detect a change in public opinion towards your brand, a negative reception to a newly launched product, reactions towards your latest marketing campaigns. For instance, if the sentiment score for a new product is negative, you can research, ask questions, and improve.
Using opinion mining across the board, you’ll find consumer pain points that can be remedied or removed. Consumers love your product, but grouch about slow delivery. Your customer support team has an excellent rep, but your order process is buggy. Monitoring consumers’ attitudes and opinions will reveal areas that should be improved to meet the needs of your audience.
Monitor brand reputation
If you understand the sentiment behind consumers’ comments, you’ll know how they feel towards your brand. Look for changes in sentiment - positive and negative - after implementing a marketing campaign, attending an event, or launching a new product.
Improve customer experience
Tracking sentiment will give your CX team a heads up. They will understand how a consumer feels at every stage of their journey. Any pain points encountered, can be addressed immediately. The customer experience can be more personal, boosting engagement, driving more revenue, and reducing churn.
Managing a crisis
Sentiment analysis will alert you to shifts in opinion. A sudden increase in negative mentions, tackled immediately, can stop an issue becoming a crisis. If you are hit by a PR disaster, measuring the sentiment and filtering the comments by severity of negativity, makes it easier to target your messages. Identifying conversation with positive sentiment means you can amplify those messages - take the edge off.
Understand how your brand compares in your industry. Monitoring the sentiment surrounding your marketing campaigns and how they affect consumer perception of your brand, in comparison to your competitors, will enable you to tweak what’s not working. And, if you see negative sentiment around a competitor’s campaign, you can steer conversation towards your brand and win a new customer.
McDonald's vs Burger King - the battle of the sentiment.
In this instance, I’ve compared McDonald’s with Burger King. 13 months of opinion mining, shows McDonald’s has 62% negative sentiment compared with 40.5% for Burger King. Quick Search - Talkwalker’s social media search engine - having analyzed the sentiment, allows you to dig deeper and find the cause of this negativity.
Looking at social media content, blogs, reviews, and new sites for the 13 month period, I found the root cause of the negative sentiment. A YouTube video was posted on October 9, ridiculing the McDonald’s brand.
Quick Search identified the cause of the slump, and gave a negative sentiment score.
YouTube video ridiculing the brand.
"Quick Search provides such an easy and user-friendly opportunity to deep dive into your competitors' social sphere; letting you harness their strengths and weaknesses to improve and cultivate a winning marketing strategy. For a specific breakdown of the importance of this, you should definitely check out Talkwalker's latest article on the necessity and implications of competitor analysis for your business and brand."
Christina Garnett | Marketing Media Maven
Finding brand influencers
Tracking sentiment will not only reveal how consumers feel, but it will help you find influencers that are discussing your brand in a positive way.
If you search for keywords in your industry, you can then find potential influencers that are discussing these topics. Find those that have positive sentiment surrounding their content, and the reaction of their fans and followers, and your industry.
Monitoring sentiment is a great way for your customer support team to track consumer dissatisfaction and then address issues before the negativity grows. If your business relies heavily on word-of-mouth marketing - review sites - it’s crucial that you’re in tune with your audience and how they feel about your brand.
If you can turn around these negative comments, appease the consumer and make them happy, you’re demonstrating to other consumers how efficient your brand is. Chances are, you’ll turn that frown upside down and win positive user generated content.
Sentiment analysis - opinion mining, emotion AI - is the use of natural language processing (NLP) to analyze online social conversations and determine deeper context - positive, negative, neutral.
And, in English… it’s a way of looking at online chat and working out whether it’s good, bad, or on the fence.
User generated content
Sentiment analysis will reveal a consumer’s opinion with regard to your product, services, team, location, ads, industry, and competitors. It’s a form of social media analytics that can improve your bottom line. The sentiment score of user generated content can steer product development, warn of an escalating crisis, and increase the accuracy of your competitor analysis.
For instance, you’ve launched a new product and you want to know how it’s being received. You could instigate a costly market research survey and spend man-hours making cold calls. Or, you could use opinion mining to understand how everyone feels.
Puma's latest dad shoe is perfect for long walks through the mall. https://t.co/pzt7L8xgSI— HYPEBEAST (@HYPEBEAST) January 27, 2018
Green flag indicates positive sentiment.
Natural language processing (NLP)
Earlier, I casually dropped ‘natural language processing’ into the mix. Also called text analytics, data mining, or computational linguistics, it’s used in social listening to identify and analyze opinions in text. It’s a computer system that processes human language in terms of meaning. It can understand that words can make up a phrase. Phrases can make a sentence. Sentences transmit messages. Following analysis, it marks content as positive, negative, or neutral.
The insights revealed, to be of any use, have to be accurate. But humans aren’t logical, we don’t all speak the same language, we use slang, abbreviations, and acronyms. We express our emotions, and we’re liberal with sarcasm. We change topic mid article. We alter our tone of voice, depending on which channel we’re using. Be it a short tweet, a 2000 word review, or caustic customer feedback. How does a machine grasp the subtle nuances that are so natural to us?
Talkwalker’s sentiment analysis feature understands the human language - 30 languages, to be precise - with up to an average 90% accuracy. It gets the sentiment, it gets the sarcasm. It’ll pull out user generated content and identify the positive and negative sentiment so you understand consumers’ preferences, what they’re passionate about, what they’re unhappy about, so you can improve customer experience, improve product development, and make smart business decisions to improve your bottom line.
Human vs machine
Teaching a machine to be human. Wow! Sarcasm, slang, and irony can still catch humans out - I’m talking to you Alanis. A sarcastic comment. Whoosh, straight over our head. Great, another sarcastic remark I missed. How wonderful. I’ll take a raincheck.
But, on the whole, we get it. For a machine, it has to be translated into objective and quantifiable scores that take into consideration the many nuances that we use when we speak. For instance…
“This guide is totally sick, the author destroyed it. Whoever wrote it is a legend!”
Aw, I’m blushing.
But, an untrained machine would think this is not a good guide, because the author ruined it. In fact, the author doesn’t even exist. The sentiment score of this statement would be negative.
Machine learning enables sentiment analysis to get more and more accurate, as algorithms learn and adapt to the commonalities in conversations and how the context of conversations can change outcomes.
When we share our opinions on social - comments, reviews, complaints - we also share our emotions. We’re passionate about a new movie, outraged at a lack of customer care, tickled pink that our flight lands on time.
As a brand, you should be analyzing these shared emotions - this sentiment - to understand what matters to consumers. Consumers that you want to love your brand. Use sentiment analysis to monitor online conversations to identify consumers’ emotions. Are they positive, negative, or neutral?
Keyword scoring, calculation based on predefined categories, and humans. Three methods adopted by most providers.
This method gives words positive or negative scores. Good = positive. Bad = negative. You see the problem?
— troye sivan (@troyesivan) October 24, 2015
The advantage of keyword scoring is that it’s quick, predictable, and doesn’t cost the earth. Rules can be applied but the results are dodgy when used too broadly, because keyword based sentiment analysis doesn’t get context, sarcasm, or slang. Plus, depending on the engineer designating the percentages of what are considered negative and positive words, the results can fluctuate.
Quick and dirty - accuracy levels are roughly between 50 and 80%. Not reliable enough to make smart business decisions.
This method has users categorizing several results - the training set. Then, an algorithm is employed to decide on future analysis. The level of accuracy is higher than with keyword scoring, but it’s flawed. Man-hours are extensive and the quantity of results are reduced.
While this method is more efficient with regard to the accuracy of the results, you don’t get much for your money. And, to get a handful of results, costs time and money.
Technology goes out the window with this method. Humans coding sentiment. The accuracy level is sick - hey grandpa, that means it’s good.
Man-hours? Ouch! Trying to catch a comment with negative sentiment and stop it from becoming a crisis, is a race against time.
Cost? Epic fail!
While the results are great, humans interpreting humans is always going to bring the best conclusion, it’s still subjective. Take into account the ever-growing social media world, and there’s way too much content for a team of humans to analyze without the aid of an automation tool.
AI-powered sentiment analysis
Our data science team is so baad. It developed AI-powered sentiment analysis that understands the full meaning behind a sentence - the true meaning. The algorithm will accurately recognize a consumer’s attitude and contextual reactions - in tweets, blogs posts, and articles.
BrandX, are you for real?
Deep learning models are used to simulate the cognitive functions of the human brain, understanding language patterns and basic sarcasm and irony - the algorithm copies how our brains understand.
To achieve a near 90% accuracy rate, our data science gurus had to classify tens of millions of results. Result!
Hasta la vista, baby!
Merck KGaA is one of Germany’s leading science and technology companies. Looking to improve product development, it turned to Talkwalker’s sentiment analysis.
One of Merck’s key products are liquid crystals - those things that manage light in LCD screens. The brand’s dilemma? Should it focus on products that bring a higher resolution and brightness, or products that improve the contrast ratio? What would consumers prefer for their smartphones?
Comparison of features of high-end smartphones by sentiment.
Results proved that resolution and brightness - along with color and battery - were critical, while consumers didn’t consider contrast to be that important.
Sentiment analysis of resolution.
Digging deeper, it became apparent that the need for high resolution screens has positive sentiment. Armed with insights that answered the question - what do customers want? - Merck focused on enhancing resolution and brightness. Leaving contrast concerns to another day.
In this instance, acute sentiment analysis on a niche topic, brought precise results. Cross-referencing the sentiment data with specific product features, meant that Merck could adapt its product strategy accordingly.
Having checked out numerous sentiment analysis tools on the market, I’m not going to waste your time with a great long list. Knowing that your top priority is accuracy, there is only one tool worth mentioning.
Ahem… say hello to Talkwalker.
Sentiment analysis isn’t rated in the social listening industry. Why? Because of all the man-hours involved, which leads to huge expenditure. But our data science team have overcome that, in spectacular fashion.
Talkwalker’s sentiment analysis technology now gives brands the ability to identify consumer sentiment with up to an average 90% accuracy. It’s so sick, I’ll say it again… sentiment analysis with 90% accuracy.
Let’s have a Quick Search look at the last three months of sentiment for Diet Coke.
Not a great sentiment score, but la di da. Let’s look at January 22, the biggest negative slump. What caused it? Is it an attack on the brand that should be addressed?
Trump sat on his hands during the shutdown and did absolutely nothing except watch tv, slurp down Diet Coke’s, cry about missing his stupid fundraiser & moan about me, Dems & #TheResistance. The time has come for Trump to resign. Everyone who retweets this agrees. #TrumpResign— Scott Dworkin (@funder) January 22, 2018
Scott Dworkin has 321K followers.
Not an attack on the brand, but worth monitoring. Let’s look at February 6.
It is laughable that John Kelly, who takes orders from a man that has a button on his desk to order Diet Coke, called Dreamers “lazy”.— Tony Posnanski (@tonyposnanski) February 6, 2018
Tony Posnanski has 108K followers.
Okay, I’m starting to see a pattern. The brand is not being attacked, but unfortunately, when your product is mentioned in posts poking fun at the President of the US, you’re sentiment score is going to take a bashing.
In January of this year, Tesla had a sudden and unexpected spike in negativity.
Looking closer, the source of negativity became clear.
An article published in Reuters, highlighting a delayed production target - the second delay - of the new Model 3 sedan.
While negativity was to be expected, finding the root-cause quickly meant that Tesla would be able to address the situation head on, before it damaged brand reputation irreparably.
Last year, BlackBerry launched the KEYone. You can see in the graph below, the sentiment surrounding and following the launch.
Let’s look at the slump on June 5.
Article published in BGR - website about the mobile and consumer electronics market.
The article reveals hardware issues surrounding the new phone, which - if ignored - would damage the brand’s reputation.
Tracking consumer sentiment during a product launch will identify all the positive and negative mentions. In this instance, sentiment analysis has highlighted valuable consumer feedback, meaning the brand can fix the technical flaws and protect its reputation.
The future of sentiment analysis
Exact science? No. But, our data science team absolutely will not stop, ever...
At Talkwalker, we understand that being able to accurately classify sentiment is essential. Brands need to be able to benchmark brand health indicators, supplement the data with demographic information, and combine product features to give consumers what they want.
Our data science team is - as we speak - working to improve the technology even further. 90% today? Tomorrow...
How do we plan, execute, measure, and analyze marketing campaigns? That’s a real hard question to answer with a single solution. The short answer? Read this eBook. The long answer? Seriously, read this eBook.
My free eBook will help you plan your marketing strategy. How to report on last year, choose your targets, set your KPIs, and monitor the results. It’s packed full of practical and actionable steps, templates, checklists, and real-life examples. There are brainstorming best practices, how to find customer insights that have real value, how to write your vision and mission statements, SWOT analysis and finding SMART goals, identifying your share of voice, and analyzing results.
Learn to listen, listen to learn.
If you’d like to check out Talkwalker's sentiment analysis tool, sign up for a free demo with one of our experts. Seriously, you won't be disappointed!