Man vs. Machine: What’s Best when Analyzing Social Media Sentiment?


By Anthony Carli and Nick Horowitz

Nearly 500 million tweets are sent every day – an absurdly large amount of content that would be impossible for any one person to sort through. But, as public affairs professionals, this incredible amount of data has become an invaluable resource for understanding what’s being talked about and who is doing the talking. Thankfully, there are numerous tools available today – and more on the horizon – that allow us to take the fire hose of social media content and funnel it down into something manageable and useful.

Popular fiction is littered with (sometimes cautionary) tales of artificial intelligence becoming more capable than its human creators. And as our world continues to look a little more like sci-fi each day, we’re watching as computers become smarter and smarter. One of the ways this intelligence is manifesting is analysis of big data – the ability to compute and pull insights from vast amounts of information.

The dark side of many of these human vs. computer stories comes when machines not only surpass their creators, but replace them. Luckily, this bit of science fiction has not yet become science fact.

In the context of social media analysis, this is especially true. While several programs seek to analyze the meaning of tweets they fall short in understanding the nuance and subtlety of human conversation. These services have very important functions – such as compiling data on impressions, identifying influencers, and pulling out statistically significant trends – but when it comes to analyzing language and the deeper meaning within a set of 140 characters, there is only so much a computer can comprehend.

While social media analysis tools have certainly improved, there’s still no true replacement for a team of informed, insightful (human!) practitioners who can take the insights gained from big data and turn findings into actionable strategies.

Take the recent speech by Israeli Prime Minister Benjamin Netanyahu to a joint session of Congress. Surrounded by contentious debate even before it began, Netanyahu’s address generated enormous conversation among influential Twitter users. A leading social media analytics tool shows that in the month leading up to the speech, users tweeted about it hundreds of thousands of times. What we can’t derive as easily is an understanding of why people are tweeting about the speech – what parts of this conversation are they invested in and how well represented are different sides of the debate. Using the same tool and the same query on the Netanyahu speech, we can see that sentiment analysis has quite a long way to go.

According to the analytics tool, 1 percent of tweets regarding the speech were positive, 5 percent were negative, and 94 percent were neutral. This stands in sharp contrast to traditional opinion polls that showed that engaged American voters were sharply divided on Prime Minister Netanyahu’s speech. Surely, social media conversation should reflect these findings. Yet when looking at a sample of 20 positive and negative tweets, the sentiments of users in the two categories are often indistinguishable from one another.

Take this “positive” tweet, “GOP inviting the Pope to address Congress brilliantly exposes Dem’s Netanyahu hatred,” versus this “negative” tweet, “#Netanyahuspeech @SpeakerBoehner If Democrats are ‘too busy’ to attend @Netanyahu speech, invite other Americans who support Israel.”

The message in these tweets is identical; Democrats are unsupportive of Netanyahu. Both users are unsatisfied with those who skipped the speech. Yet one was labeled as positive, and the other as negative. These examples demonstrate that algorithms still have work to do to provide an accurate read of public opinion.

Perhaps someday computers will reach the point where they can truly parse human conversations on these scales and grant us instant and accurate insight into popular thought. However, that day remains firmly in the future with warp drives, ray guns and hoverboards. For now, good old fashioned human beings (coupled with the right tools and experience) remain the most reliable and accurate resource to gain insight into the nuances of online conversations.

Leave a comment

Your email address will not be published. Required fields are marked *