Marc Goldberg is a contributor for the Trust Web Times. Marc is also the Chief Revenue Officer of Method Media Intelligence an MRC accredited ad verification and measurement company.
Rick Bruner is CEO of Central Control
Rick and I exchanged emails for this interview and it has been slightly condensed and edited
Marc Goldberg: Rick What is a research wonk and what is The Research Wonks?
Rick Bruner: The word “wonk” comes from “know” spelled backwards. Websters defines it as “broadly : NERD.”
I set up “Wonks,” aka Research Wonks 16 years ago as an email group for specialists in advertising and media research & analytics. Since then it’s grown into a vibrant professional forum, with many of the sector’s business leaders taking part in regular substantive discussions of business issues, sharing of resources, jobs networking, and more.
In the last year, with help from a colleague, Stefan Tornquist, we built new features, including a website, with members-only directory and archives, and public listings for jobs, resumes, events, branded content and featured companies.
Now, we just launched a podcast, Measured Response, likewise focused on media and advertising measurement, ad-tech and data science. I’m very glad that you and Method Media Intelligence stepped up to be our first podcast sponsor! Thanks again. The episode also features an interview with my friend Ari Paparo, co-founder of Beeswax and newly Head of Partnerships and Strategy at FreeWheel, since his firm’s recent acquisition by Comcast.
MG: Ok Nerd, is Wonks free to join?
RB: Yes, for qualified applicants working in media/marketing measurement, market research, analytics, etc. Apply on the homepage of ResearchWonks.com.
MG: This list was started because you are a researcher, and you created it for your research cohort, is that the right word? Tell us about your story and on how you got here.
RB: Yes, points for “cohort.”
I was a journalist at the start of my career. In the late ‘90s, I lived in San Francisco and was reporting for Ad Age, about the birth of internet advertising. After that, I reinvented myself as a research guy, and a few years later ended up running DoubleClick’s research department. Google moved me into a similar, larger role. Later I had team leadership roles at Marketing Evolution and Viant, after which I set up my own business, Central Control, Inc.
My insight — obsession? curse? — is that many marketers are missing a key ingredient in their ROI and optimization analytics, namely the highest-grade of evidence of what’s working that comes from experiments.
I’d summarize the problem as this: The “scientific method” is essentially observation + experiments. But most marketers, particularly in terms of paid advertising media, rely too little on high-quality experiments to validate the many assumptions in their observational analytical systems, such as MTA and MMM. They substitute the correlations that are identified in the observational models (i.e. “XYZ tactic ‘works’”) as if they were evidence of causal effect (instead of hypotheses to be tested, which is what they are), and they base big spending decisions on those correlations alone. The result is all sorts of degrees of wrong.
This is what a lot of people mean by the “incrementality” movement. The two-part podcast from Freakonomics on “Does Advertising Actually Work” is worth a listen in this regard. This Harvard Business Review piece is a great read on the subject: “Marketers Underuse Ad Experiments. That’s a Big Mistake.”
MG: So Central Control answers the question that Wannamaker asked?
RB: Since you asked, yes. But, really, this time. What’s funny to me is that advertisers are glib to accept half their marketing investment is misspent, but they are loath to spend much money to fix it. Do CFOs even know about this joke?
The cliche I prefer is “correlation does not imply causation.” Without experiments — preferably the best kind: randomized controlled trials (RCT) — advertisers really can’t know what is working, literally “because science.” All manner of statistical approaches that marketers use to inform spending decisions, based on trended “observational” data, including MTA, MMM, clicks, machine learning and more, are all subject to forms of bias that can result in highly erroneous conclusions (e.g., a certain tactic which appears great at “driving conversions” is actually a negative-ROI money pit simply targeting sure-thing converters, providing no incremental lift).
In pharmaceutical production, randomized controlled trials are required by law to demonstrate to efficacy. Advertising is not a matter of life and death, thank goodness. But market share is at stake for advertisers. Brands slower to adopt this incrementality trend risk losing customers to more progressive advertisers that are getting more bang for their buck, proving causal advertising effect and fortifying observational models with more and better-quality experiments.
MG: Help folks understand more about incrementality and how you are able to show it?
RB: Let’s take apart the phrase “randomized controlled trial.”
“Trial” here means experiment.
“Controlled” means there is a control group. The control group, which does not receive the “treatment” (i.e. exposure to the ad campaign), allows us to measure the true “incremental” contribution of the ad campaign on an outcome like sales. This is what scientists call the “counterfactual,” as in “How much money would that same targeted audience group have spent on my product if I’d kept all my campaign dollars in my pocket instead?”
“Randomized” is the key. You take the audience for the campaign and randomly assign members into the test or control group. (That’s the tricky part.) The randomization is what makes it the “most unbiased” form of causal measurement. The control group if it really represents the same “cohort” (if you will) as the test group, represents the “counterfactual” of what-if-I-hadn’t-run-the-campaign better than any other method.
That technique makes it very hard for correlations to confound the measurement, very hard for fraud or algorithms to mess with it. It’s not a silver bullet. It’s hard to get right. But it’s worth doing, if advertisers want to stay at the front of their game. It helps fortify the MTA and MMM and other observational systems with truth-set data to calibrate assumptions in the models more accurately.
You can do it for linear TV (with live addressable), OTT, CTV, search, walled gardens and many forms of media. It’s harder in display, because of the identity issues, but also doable there.
MG: You have been on the research side for a long time and in digital since 1996. What is one thing that we got wrong and you can back in a time machine and fix?
RB: Oof. Just one?
I’d say cost-per-click. The irony is that it was Procter & Gamble that foisted that one on us, before Google, before GoTo/IdeaLab, it was a directive from P&G to take it or leave it, as they perceived it to be “more accountable” somehow, and reluctantly as hell, a bunch of leading digital media companies at the time, like Excite, buckled and took them up on it. It led to clicks being inextricably at the heart of most “predictive algorithms” for ad targeting. Problem is, they’re not predicting something that maps very well to true advertising effect, if you measure it right.
I had a big Twitter rant on the subject a few months back.
MG: Do you have funding? Are you looking for funding?
RB: We’re working with a number of big advertisers and other clients and aren’t looking for funding presently.
MG: How has quarantine life been for you? Professionally and personally.
RB: Professionally, with the disruption to the economy and business as usual, and the radical changes in historical trends of consumer buying behaviors and media behaviors out the window, it has dialed up demand from advertisers looking for more certainty in their marketing spending.
Personally, it’s been quite a year, I think we can all say. We are very grateful for our circumstances. We’ve stayed healthy and mostly sane, myself, my wife and son. We own a house in Brooklyn, and though the spring in NYC was harrowing, we at least have a backyard and more space than many NYers. Brooklyn and Manhattan are different worlds in COVID times. There has been still a lot of vibrancy to life on main streets and neighborhoods in Brooklyn throughout.
At least until our endless winter started. I’m very ready for that to be over. I’m also very eager for my turn at the vaccine. I’m optimistic for much relief from pandemic stress by the later part of the year.
MG: Imagine a world where we are all back. What would you rather do:
Do a presentation in a conference room with people or
Stand in line for popcorn at a movie theatre.
RB: I like the movies, and I’ll be eager to go. We’ll probably start with some family-friendly blockbusters.
But I’ll give a shout out here to my peeps at I-COM: getting back to Europe, bringing the family with, seeing all my beloved I-COM family members, talking digital ad measurement with the best in the world, and getting at tax write off? Are you kidding? That will feel like life being back to Before Times for me!
MG: Again, world is back and you can now go to concerts. What concert would you like to attend if they were playing in your area?
RB: I always regret taking a pass on the big Boomer rock bands when they do tours, like The Stones, I kick myself for missing. I’ve never seen Dylan. I’d go for that, for sure. I’m Gen Xer, but already so many of my punk icons are already gone.
Radiohead always delivers a great show. I need my buddy Tom Burg to hook me up; I go where he tells me.
More than big rock concerts, I’m looking forward to introducing my son to Classical concerts and other types of the “high art” stuff NYC specializes in, now that he’s old enough to appreciate it. That and local live music, small-room jazz, the neighborhood scene, musician friends’ shows and that kind of thing, which has more been my speed in recent years than big rock concerts.
I’m a hack musician myself. It’s such a social experience. I am really craving getting back to listening and playing and supporting the arts and social graces they offer us.
