Operations Research Valuable in Netflix Prize

Netflix has been running a contest to see who could significantly improve its ability to predict how much someone will like a movie. Lots of high-powered (and less so) groups have been applying huge amounts of scary math to reach the goal. But, as Wired reports, someone suddenly appeared out of nowhere last fall, approaching the few in the lead and making progress at a rate that took them months. What I find amusing about this story is that the guy isn't primarily a mathematician. Instead, he got an undergrad degree in psychology, a masters in operations research, and is a retired business consultant.

His trick? He's taking human psychological factors into account rather than treating this as nothing but a numbers game. I think this is a great lesson. Too often companies want to reduce people to numbers, and, certainly, you can look for mathematical patterns in group behavior. But sometimes the individuals will start acting in ways you never expected, demonstrating sudden shifts in behavior. The math doesn't explain it because the math is looking at the results, not at the causes. What might be predictable if you could follow the emotional contours of people turns into a jarring lurch if you're sitting blindfolded in a car and couldn't see the dip in the road before the auto dropped down. Will most companies learn anything from this? Probably not. They might nod and say, "Yes, we really should focus on psychology," but they don't grasp this in a practical way in how they do business today. And given that areas like marketing are pure psychology, if they don't get it by now, they never will.

Google Wants to Know Everything - About You

According to a piece in the Financial Times (I'm not sure how this got to be a free link, but who am I to complain?), Google wants to know everything about everybody. Literally:

Asked how Google might look in five years’ time, Mr Schmidt said: "We are very early in the total information we have within Google. The algorithms will get better and we will get better at personalisation. The goal is to enable Google users to be able to ask the question such as 'What shall I do tomorrow?' and 'What job shall I take?'"

There you have it - Google with the gloves off. Aside from questions of privacy, civil liberty, and propriety this naturally raises, there are some serious business issues.

To the degree that Google succeeds, it puts virtually all businesses in its debt. Depending on how it decides to analyze and answer such questions, it directs people in their choices. If your company is on the outs with Google, might there be a chance that it freezes out your chances with customers? Will you have to - eventually - pay a fee to remain in the running, if not with the current management, then with a future team? People who think that all the tumult over Google and Yahoo and Microsoft is solely about online advertising aren't looking at the bigger pictures. In a society where an increasing number of people want to be told what to do, having an ever-increasing collection of information on them as well as an ability to be a front end to the Internet means controlling business beyond what can happen even in a monopoly.

TV Advertisers Get Taken By Their Own Strategies

The New York Times Magazine had an issue devoted to middle age, and one of the article was called TV's Silver Age. Part way in there is the following paragraph:

Every Wednesday morning, newspapers across the country run a chart of the previous week’s highest-rated television shows. Most television executives basically ignore that list. They have eyes only for subsets of those overall figures, particularly one they call “the demo.” That’s televisionspeak for viewers ranging in age from 18 to 49. The demo may seem nonsensical — after all, what does a high-school graduate have in common with someone becoming a grandmother for the first time? — but it drives the television business.

If your company does or even contemplates television advertising, read that paragraph and the three that follow. They lay out the following history of ad buying:

1. Nielsen audience ratings were once broadly based.


2. ABC, being last in the rankings, decided - strictly as a marketing tactic for itself - to emphasize the 18-to-49 demographic because the network was attracting younger audiences and it looked good.


3. Further, ABC argued that the baby boom generation was vital, because it was used to televion and was huge.


4. Advertisers bought into this and the special reports that Nielsen generated for ABC became the important numbers.
   
5. As demographics shifted, advertisers never got it through their heads that the old arguments for skewing young had less and less credibility.

As a result, the advertiers still look for young audiences without ever considering where the most consumer dollars are available to be had. If you've read through this far, here's the conclusion: folks, you have just been taken for, oh, about 20 to 30 years and have largely been wasting the money your companies entrust you to wisely spend.

How do I, a non-advertising expert, figure I can say this? Because I'm willing to look at the numbers. Now some good news: just because you've done everything one particular way for so long doesn't mean that you can't switch strategies. Here are some steps to at least start moving in the right direction:

1. Redefine your market segments. Make advertising segmentation match your customer segmentation as much as possible. If advertisers refuse, mutter "online ads" and the names of competitors, and that should make their spines bend over backwards.


2. Do some research to show how much per capita spending in your category happens in each of the segments.


3. Create a segment weighting factor. If, for example, 30-to-40 year olds buy represent 40% of your sales, the factor would be 0.40.


4. Compare average spending by your customers to the appropriate segment spending as a ratio. The higher the ratio, the more invested in your company the cusotmers are and the more important they should be to you and where you might be looking for more.


5. For any program, multiply the number of audience members in a given segment by the average category spend. Now you have a first approximation of how much money there is for your type of product watching that program.

Sure, "real" analytics would get far more complex, but you don't need to have a Ph.D. in mathematics to start understanding what is going on. Don't get snowed; get smart.