Operations Research Valuable in Netflix Prize

Netflix has been running a contest to see who could significantly improve its ability to predict how much someone will like a movie. Lots of high-powered (and less so) groups have been applying huge amounts of scary math to reach the goal. But, as Wired reports, someone suddenly appeared out of nowhere last fall, approaching the few in the lead and making progress at a rate that took them months. What I find amusing about this story is that the guy isn't primarily a mathematician. Instead, he got an undergrad degree in psychology, a masters in operations research, and is a retired business consultant.

His trick? He's taking human psychological factors into account rather than treating this as nothing but a numbers game. I think this is a great lesson. Too often companies want to reduce people to numbers, and, certainly, you can look for mathematical patterns in group behavior. But sometimes the individuals will start acting in ways you never expected, demonstrating sudden shifts in behavior. The math doesn't explain it because the math is looking at the results, not at the causes. What might be predictable if you could follow the emotional contours of people turns into a jarring lurch if you're sitting blindfolded in a car and couldn't see the dip in the road before the auto dropped down. Will most companies learn anything from this? Probably not. They might nod and say, "Yes, we really should focus on psychology," but they don't grasp this in a practical way in how they do business today. And given that areas like marketing are pure psychology, if they don't get it by now, they never will.