This story is about the surprising connections between a mediocre high-school play, a prestigious dental researcher, and your business’ marketing strategy. And if that’s not enough, there’ll be a treat at the end.

First, the high-school play. My brother, the good-looking one, played the male leads in our high-school plays, while I got the character roles. In short, he was kissing girls while I was eating flies.

Good life prep, that.

One of the plays he starred in was The Rainmaker. No self-respecting high school puts on The Rainmaker anymore, because it’s not much of a play. Katharine Hepburn made it some t hing in the movies, but Katharine Hepburn could make a NAPA parts catalog unforgettable.

(It also became of the few Tanya Tucker hits that didn’t involve someone going crazy and walking into the muddy Mississippi dressed like Scarlett O’Hara.)

The premise of The Rainmaker is that a town needs rain, it hires a rainmaker, and before he leaves, it rains.

Note how I worded that: It rains while he’s in town. He claims to have made it rain, but there’s no proof that he did anything except kiss a repressed spinster.

Now, for the dental researcher. Dr. Fred Eichmiller is the dental director at Delta Dental of Wisconsin. Prior to joining DDWI he’d been a renowned dental researcher at the Centers for Disease Control and Prevention, and he’s still one of the world’s greatest experts on the narwhal, that seal-like thing with the unicorn horn sticking out of its head.

(That’s not a horn, BTW. It’s a tooth. That’s why a dental researcher pals around with sea creatures in the Arctic Ocean.)

We’d walk into Dr. Eichmiller’s office and get his take on the latest news proclaiming that teeth cleanings can prevent Alzheimer’s, but he was always one step ahead of us. He’d seen the story and read the research, and he’d send us off with the cautionary reminder that correlation is not causation.

Just because two events happen at roughly the same time or in roughly the same order doesn’t mean that one caused the other.

There are many other possibilities: They could both be side effects of a root cause, they could be part of a larger network of effects and counter-effects, or it could just be coincidence.

Correlation-is-not-causation is why a study that shows fish oil helps prevent heart disease can be contradicted several years later by another study that shows the opposite.

Correlation-is-not-causation will also eventually bring down the rainmaker. His appearance may be correlated with rain, as it was in the play, but he didn’t cause it to rain. Sooner or later, the correlation won’t be there, and he’ll be skint.

More prosaically, and closer to home, this mistaken thinking can be disastrous for your business’ marketing.

If you assume that something you did from a marketing standpoint caused a business reaction, and it turns out they were merely correlated, you could be in big trouble.

(Perhaps a) cause and (maybe an) effect

So let’s suppose you make a whole bunch of changes to your website based on an SEO audit, and your page ranking goes up.

Great! Google is looking upon you with favor and granting you peace, and the sky – or page one – is the limit.

Not so fast, bucko. You’re looking at this through you-colored lenses. You made changes, but what did your competition do? What pages aren’t ranking above you that used to be – and when did those pages change their copy? Did someone go out of business? Did a news story move down as its freshness waned?

You’re assuming your page-rank improvement was caused by your changes, when it could just as easily be correlated with that. And the second a competitor refreshes their site or another news story crops up, you’re back down a hole.

Sorry, but correlation is not causation.

Are there other instances where a change in marketing is assumed to cause a change in business? Scads. A new ad campaign launches. An email goes out. You start using a different social-media channel. You redo your collateral materials. You name it, and you’ll want to say it caused a business outcome.

So how can you really be sure that A caused B? Do your research.

Here are some rules to guide that research.

Don’t assume one measurement can tell the story

Unless you’re doing marketing where only one measurement matters – for instance, a Facebook campaign where you’re giving people a product sample for liking your page – you’re probably going to see a decreasing amount of correlation the more measurements you add to your analysis.

That’s good; if the actual amount of causation is smaller than you initially thought, that will help you make better business decisions in the future. On the other hand, if all the measurements point strongly in one direction, you can feel safer in stating that A caused B – and that’s what you were after in the first place.

Ask people what happened

All attribution models are flawed. However, if you ask people questions about their behavior, and keep asking them over time, you’ll eventually create a platform that’s largely free from self-report bias to assess actual changes in behavior and attitude.

People may tell you A caused B without really knowing whether it’s true, but if you construct your research properly, you’ll be able to determine causation from everything they say, not just the answer to one question.

Do it again, and see what happens

That’s what scientists do. If you can afford to repeat the same thing, or if it makes sense (you wouldn’t want to launch another new website after you just launched one, for instance), repeat the experiment. Keep the conditions as close to identical as you can. In the business world, it’ll never be a perfect replication, but it will help confirm or reject any broad conclusions you drew the first time around.

A/B test – the right way

Marketers love to say, “Oh, we can A/B-test that,” and then they do some wild thing where 15 different variables are changed from A to B.

That’s useless; don’t do that. Instead, change one variable – two, at the absolute most. If you’re doing a test e/mailing, mail to lists that are nearly identical in every demographic category. If you’re A/B’ing banner ads, run them on the same types of sites in the same positions at the same frequencies.

Once you’re done with the A/B tests, follow up with research that hits on the differences between A and B and see if anyone noticed.

Do everyone a favor: If you’re going to A/B test, take off the you-colored glasses. You may not like what you see, but that’s the point.

See what your competition does

How your competition reacts to your changes will tell you more than practically anything about their real impacts. After you change your website, do they change theirs? Do they react to your SEO campaign with one of their own?

We could tell a newly launched blog and social campaign was hitting home when a major competitor started copying our blog topics and social voice. It was no substitute for the ultimate measurement – sales – but we knew we weren’t being ignored.

Hey, everyone wants their ideas to work. I want your ideas to work. But I also want to make sure they’re really working, because if they’re not I have another idea.

Also, I promised a treat. I can’t give you real cookies or candy, but I can give you my secret family recipe for holiday toffee. It’s right here.

If you gain five pounds after consuming a whole pan of my toffee, remember that correlation is not causation. Except maybe in this case.

Repeat After Me: Correlation Is Not Causation