Reasons Managers and Business Decision Makers MUST Understand Basics of Research

Stop Bandwagon Jumping With These Eight Steps

It's a faddish world. Things come and things go, influenced by the Buzz, what seems to be popular, and human tendencies to cognitively distort the information they take in.

Quality circles, TQM, learning styles, personality styles are all good traditional examples of fads, some of which have not yet expired, that have been adopted by business decision makers in the past, often to be discarded after "someone" notices the decisions were based less on data, and more on a "gut feeling" or simple "common sense", the latter not being terribly common and often not sensible.

Now it's worse. Social media has changed the landscape of informed decision making, making it possible for anyone to say anything, and to promulgate the APPEARANCE of validity of the current popular thinking.

The problem is that by the time it is discovered the emperor is stark naked, millions of dollars end up misrouted to things that upon reflection, and a bit of knowledge, would have seemed silly in the first place.

Three important areas have become completely contaminated by a lack of knowledge on the part of business decision makers, and a gullibility spawned from mental distortions that push us to adopt what is popular -- i.e. popularity = truth.

Specifically these areas are: customer service, social media as a business tool, and learning/e-learning, three areas in which I have some expertise, and follow.

Case In Point: Customer Service and Social Media

Today, and as I speak, people are tweeting and retweeting about a "research study" that "proves" that companies must attend to what people on Twitter are saying about their brands. I've seen about twenty retweets of the same information so far, and no doubt there's hundreds more. The numbers are just silly. Based on a terrible biased sample, the claim is that: "More Than 1M People Per Week View Customer Service Tweets" and, another finding was that there was a huge deluge of tweets of the complaint variety (sorry, the tweets have scrolled away, now and can't be found so I can't provide the quote).

The problem is that even with a superficial skimming of the information provided (the original article provides little detail, but enough detail), it's clear that a) the sample is too small to mean anything; b) the "researchers" either out of incompetence or intent, sampled in such a way as to guarantee the result they wanted.

Not only that but the study is direct opposition to other, previous research that used a sample size 100 times larger (2,000 tweets versus 200,000).

The earlier and superior research concluded that of the people who talked about brands on Twitter (and the number of such tweets was fairly small, less than 20%) there were MORE positive tweets than negative.

Yet over and over companies are urged to protect their brands on Twitter, and use Twitter to counter-act negative tweets.

If decision-makers get their information from Internet sources (and who doesn't) it would make sense that they are swayed by the simple ubiquitessness of wrong information, sent third and fourth hand by people who lack the basic wherewithal to make sense of the "research".

There's more.

Research Firms With Bias and Undisclosed Agendas

Research firms in the three fields I've mentioned have a vested interest in getting more and more people to jump on the bandwagon. That's because a) they sell "research" reports to large companies to help them implement new e-learning, customer service and social media strategies. The cost of the research reports can be up to $5,000 per report, clearly aimed at the larger corporate clients.

Not only do they sell the reports, but they tend to offer webinars, and other low overhead, HUGE margin services to help corporations get onboard.

Ever wonder why you NEVER see a major research firm do research that finds that companies SHOULDN'T be jumping on the bandwagon. Have you ever seen one of these companies conclude that it isn't worthwhile to spend hundreds of thousands of dollars on social media, e-learning solutions, or customer service solutions?

It's not that it's intentional, though it could be. It has to do with the culture in these companies, where research is a means to make money, and that's NOT how research is supposed to work. And for good reason.

You can no longer trust any research coming from independent for profit companies. I've found false conclusions, poor methodologies and incompetence in major companies (you'd know the names) in all three fields.

They NEVER talk about the limitations of their studies. They almost never cite research done by others. The research is never peer-reviewed or validated. It's junk science.

The Social Media Problem

We tend to believe that which is repeated, and that which is popular. That's the psychological basis of a lot of marketing campaigns, repeated over and over again. Marketing professionals know that the more you see a set of facts (actually claims), the more you tend to believe them.

Social media, particularly Twitter, but also LinkedIn, involves repetition. If you spend any time at all on either you will find that most comments tend to be lacking in original thought and content. Of course on Twitter you might see the same link, or the same contention retweeted hundreds of times, particularly in the customer service and social media topic areas.

It seems like everybody is convinced. So, are you swayed? Probably. Get on board, or you'll be left out.

If those writing the original source, and the retweeters could be assessed as to their competency in making a judgment about what is being repeated, and you could verify and trust them, we might be OK. But you can't.

There are no entrance requirements to write a blog, or tweet a tweet. In many respects you get influenced by people who know little more than you do.

The Seduction of Numbers

When you see research numbers are you more impressed than if there are no numbers present? Probably. We are both addicted to numbers, and don't understand how those same numbers are derived. Unfortunately, the most dangerous bad information out there contains numbers and alludes to research, almost always from people who have not read the research.

Yet you see the numbers. Numbers can't be wrong. The are impressive. Sadly it's Garbage In, Garbage Out, and the rinse and repeat.

The upshot is that lacking the ability to read and understand research, business decision makers are making poor decisions, based on bad (but nice sounding) research. And that's hurting companies each and every day.

What To Do?

1. Managers and decision makers MUST learn the basics to understand how one goes about doing simple research and how to ask the correct questions about research methods, sampling and data analysis. It's easy. Not rocket science at the level most of the research is done at. Give me 3 hours and I could explain the basics so well that any reasonably intelligent manager can learn enough to know when to discount research numbers when warranted.

2. Ask for and look at primary sources. It's not enough to read a BusinessWeek article that mentions research findings. The writers don't know how to interpret data either, and they rely on the conclusions of the source. YOU, however, with your basic knowledge of statistics and research, can go to the original research (if it's available), and quickly determine whether it's incompetently done.

3. Look for material in peer reviewed professional journals. Peer review ensures that there is at least some measure of critical thinking applied to claims and research BEFORE it's published, and after the fact as others attempt to poke holes in the studies. That's a GOOD thing. It's not perfect, but it's better.

4. Keep in mind that even if a study is done flawlessly, a single research endeavor means nothing. Truth is "approximated" through replication, repeating the methodology and improving on the research design. ONLY through looking at a collection of research over time, with multiple retesting of hypotheses, can we have some confidence in the collective results.

5. Turn off the Internet, and get your wisdom elsewhere. Why? The search engines are now incorporating "social signals" (i.e. popularity) in the system that serves up your search results. That means that what appears on the first pages, or for that matter on any page, is there partly because Google and Bing are using popularity to place them. Quality is no longer a criteria, particularly because machines (using algorithms) decide what goes where, and they are not good at assessing quality.

6. Beware of conferences where the bulk of presenters and attendees share a common bias on the topic of interest. For example, if you attend the annual ASTD (American Society For Training and Development) you will find many many people who "believe" in e-learning or other applications of technology to learning. If you go to a customer service conference, attendees and presenters will have already decided on their common opinions about the value of WOW, and of using Twitter for customer service, even though there is significant data to suggest it doesn't work that well, and can't work that well.

If you go to a social media conference you aren't going to hear that unless you have tens of thousands of ACTIVE followers, almost nobody will read your tweets. Or that the numbers about Twitter users are grossly inflated by millions of abandoned but still counted accounts.

7. You must become self-sufficient in assessing claims, or you have no choice but to do what everyone else does. Whether you learn the basics (and you should) or you hire someone who is really good, you have to have in-house people who will challenge everything. Business decision makers MUST have critical thinkers who will "speak truth to power". The more critical analysis, the better. Hire them. Retain them, and acknowledge the value of their contributions, even if and when they are proved wrong.

8. Finally, be a mini-scientist. Start out believing in the null hypothesis, meaning that you seek out information to disconfirm the common wisdom about something. In fact, identify your own personal position, pro or con, then seek out sources to prove yourself wrong.

Conclusion:

For both small companies and large, the economic climate does not allow survival if too many mistakes are made: too much money, time and resources funnelled into the latest fads.

The ONLY way you can make informed decisions is to learn to analyse and interpret research and data, and NOT going along with every Tom, Dick and Harry who can tweet numbers.

PS. Visit the "Almost Everything You Thought You Knew About Human Behavior is Wrong" where I'm building up a resource base that addresses how the brain tends us towards easy solutions, and even gullibility.