The fact that “small data” still has value prevents marketers from fully optimizing the “big data” opportunity. In fact, “big data” requires a whole new approach from marketers and marketing analytics professionals – where the methods and approaches to small data will no longer work or suffice.
“Optimize big data in your business or you will fail,” said featured keynote speaker Viktor Mayer-Schonberger at NCDM13: Where Marketing Meets Big Data” this morning. His challenge to all of us as marketers, businesspeople and citizens did not fall on deaf ears, as he was named repeatedly on Twitter (#NCDM13) the “best big data speaker ever seen.”
To optimize in a small data world, he said, we all took shortcuts because all the data was not available or not possible to process in real time. We relied on samples. ” However if you have all the data available, then you can let the data speak for itself,” Viktor said.
The advantages of sampling are greatly diminished in a big data world, he said. “Sampling requires us to choose what details we are interested in, before we do the analysis. Suppose I could change this? Suppose I could decide later what details I am interested in? Suppose I could zoom in on other details because I have the data that is available?” he asked. “That would be pretty powerful.”
Letting the data speak allows us to move beyond defined notions of correlation and causality, he said. To allow emergent ideas that come from the data be used to not just identify the correlation that we suspected, or one that even makes sense, but to identify the best correlation, he said.
This is a new paradigm for all of us. Big data is messy. It does not follow prescribed formats or traditional channels of thought, he said. Citing a Google Translate example – and why that product has succeeded where small data attempts to translate languages did not – he said that Google found that having vast quantities of imperfect data was more successful than having small quantities of high quality data. That is a new paradigm for product innovation and marketing insights, he said.
His fundamental message is that it doesn’t matter “why” there is a correlation. He cited the classic PopTarts example from his book, “Big Data: A Revolution that will Transform How We Live, Think and Work.” In this example, sales of PopTarts rose when there was a storm coming in Florida. Does this mean PopTarts sales predict storms? No. However, it doesn’t matter why sales go up, Viktor said. It only matters THAT it happens, and so WalMart moved the PopTarts to the front of the store and sold even more.
“It is comforting to us humans to try to explain the effect, to try to understand reality,” he said. But, it really doesn’t matter – you can optimize using Big Data without understanding why.
After noting many examples of how big data has revealed insights that are incredibly powerful, Viktor noted that with this opportunity comes great responsibility. There are plenty of “big brother” questions about privacy and notice and choice that come from the new uses of “big data.” “We must continue to explore this with an appreciation of the power we wield,” he said. “We must view this opportunity with humility…and humanity.”
## ## ##
Join the DMA marketing analtytics community at NCDM15 at the fabulous Palmer House Hilton in Chicago in February 2015. More info and to register on our website.