A colleague from the media industry mentioned the other day over lunch – ‘the difference between search and social is that search is now a science. Social isn’t yet’. I couldn’t possibly agree more. But this triggered a completely different question in my mind – How scientific can social really get? The answer to this is short. Very.
Fortunately or unfortunately, social media has become a bedrock for entrepreneurial activity in the past decade and the result is an amazing variety of tools and methods to measure and monitor ‘consumer generated media’. ‘Fortunate’, because this has brought about great innovation, more visibility and wonderful talent into the market. ‘Unfortunate’, because it has brought out half-baked products that are trying to set standards in an industry that thrives from destroying all standards. But the common thread to all of them is that they are scientific, quantitative methods to measure fluid, qualitative data.
Take listening tools for instance. There are tried & tested tools like Nielsen BuzzMetrics, which has literally been around almost as long as social media has been, in many parts of the World. Then there are other very robust tools like Radian6, Meltwater, SM2, etc. And then there are the ambitious ones. Almost all of them can tell you the exact number of times your brand has been mentioned or how the volumes trended on each day of a month. However, they will struggle when it comes to analyzing sentiment. Industry experts will probably put the accuracy levels of tools to measure sentiment automatically, at around 70-80%. However, I feel it could be even lower. And it does not surprise me, because things like ‘passion’ and ‘sarcasm’ are extremely difficult to interpret through an algorithm. Which is why most of them play it safe and use categories like ‘neutral’ and ‘mixed’, quite liberally.
Let us do this through an example. I searched for ‘Harry Potter review’ on Google, under ‘forums’ and landed at an interesting review of the film. One of the comments on the post was that “The reviewer himself is bitter about HP7”. Now, if a listening tool were to pick this up automatically, analyze it using Natural Language Processing (NLP) techniques and slot it into a pigeon hole, that hole would likely be ‘Negative’. This is because the word ‘bitter’ has been used in close proximity with the brand and there are no other words in the sentence to justify any other sentiment. However in reality, the person who posted this message could have actually loved this film and may have been just showing displeasure at the reviewer himself. Quite possible.
Take the ‘social influence’ tracking tools like Klout or PeerIndex. Their methodologies seem to be reasonable and they do give you some clarity in this space, which is otherwise largely grey at this point. But the question is, as you go forward, will you be able to manipulate your Klout and PeerIndex scores? If you were to follow every other person you come across on Twitter (with the hope that some of them will automatically follow you back) or if you were to take a week off from work, cherry pick the most interesting links on the web and tweet them (with the hope that you will get a lot of retweets), will you be able to move your score up? Quite possible. A Quora post by the PeerIndex CEO recently said that “our ranks are ranks which follow a log distribution…. So it would be wrong to say that some with a score of 50 has double the impact of someone with a score of 25. The right way to use the data is to apply a threshold and some commonsense.”.
The point is that social media – related tools, by the nature of the beast, have to be in Beta permanently – just like your social media strategy itself. It is definitely not an option to ‘not use’ them, because your competition has moved on from relying purely on traditional, ‘asking’ techniques. At the same time, it will also not make sense to rely entirely on them. The sweet spot lies somewhere in between, where you are relying on them for larger trends, by setting thresholds and not taking them to be absolute measures. After all, Apple cannot be deciding KPI’s for its marketing team, based on how a tool interprets this video below!