Closing The Creative Loop: Informing Art with Science

With just the scroll of a thumb, mobile News Feed allows people to discover the latest information about everything from their friends, family and favorite brands to topics in the news, virtually anywhere, at any time. The quick pace, combined with the volume of content, makes mobile a different and truly revolutionary marketing medium.

Great ads are designed for how they’re consumed. And for traditional mediums like TV, out-of-home and radio, it’s taken years of testing and many iterations to figure out what good creative is. For the mobile feed environment, we’re beginning to unearth these findings as learnings on ad creative continue to advance in this new space. Though each of these channels is different, one fact remains constant: creative is king. As an Ipsos study found, creative quality determines 75% of impact as measured by brand and Ad Recall scores.1

Ad creative is in a marketer’s control. And Facebook continues to work with marketers to test and measure which creative elements positively impact campaign objectives in News Feed. We recently spoke with Facebook Marketing Science researchers Hannah Pavalow and Kumi Harischandra, who presented their creative research at the Advertising Research Foundation Audience Measurement Conference on June 13 in New York. Edited excerpts of our conversation follow.

Your presentation looks holistically at creative measurement. Based on your experience, what is the relationship between measurement and the creative process?

HannahHannah: I’ve seen that, whether at an agency or in-house at a brand, creatives and marketers are playing with their ad executions on Facebook. But we are also seeing a gap between how they approach creative design and the actual campaign objectives they’re looking to achieve.

The ideal goal is to inform a brand’s creative process and measurement from the campaign’s objective at the start. By taking the artistic and experimental approach and adding measurement, we can understand how ad executions perform in-market and take that direction to inspire future ad creative. We’ve come to call this process “closing the loop.”  But that loop is often broken, and the creative teams sometimes don’t get any feedback on a creative’s in-market performance.

KumiKumi: Understanding the relationship between ad creative and measurement has become a priority for us. We first started sharing our learnings on ad creative at ARF Audience Measurement two years ago. A big part of our new body of research focuses on understanding what creative elements of ads drive brand resonance for marketers through retrospective and in-market research.


Considering Facebook Marketing Science’s past research
on ad creative and your new findings, what have you learned so far about what makes good creative on Facebook?

KumiKumi: This has been a continuous learning process for us because consumption on feed is so different from traditional marketing mediums. One of the research pieces included in our presentation is the retrospective analyses of both static and video ads. The main goal of the analyses was to understand the association between different ad creative and campaign results for outcomes that marketers care about.

We rated them on seven elements for static ads:

  1. Noticeability: This ad would grab your attention.
  2. Focal Point: The ad has one obvious focal point.
  3. Brand Presence: It is easy to identify the advertiser in the ad.
  4. Brand Fit: The ad fits with what you know about the brand.
  5. Interesting Information: This ad has interesting information.
  6. Emotional Reward: This ad appeals to you emotionally.
  7. Call to Action: This ad urges you to take a clear action.

We found that ads that fit a brand’s personality and ads that were noticeable had a positive impact on Ad Recall. Interestingly, we also found that ads with a lower cognitive load, both in terms of emotion and information, also had a positive impact on Ad Recall.

For video ads, we found that ads that showcased the product or service early and often were positively associated with Ad Recall. Another interesting insight was that video ads with conversation had a positive impact on Ad Recall. More research needs to be done here, but we think that conversation piques interest and draws people in.

These analyses further exemplify a fact we’ve all come to realize: Consumption on feed is very different. It’s fast, it’s frequent and it’s often done with the sound off.

What do these retrospective analyses mean for advertisers when developing creative for News Feed?

HannahHannah: As Kumi mentioned, a retrospective analysis allows marketers to better understand which creative elements are impacting their desired campaign outcomes.

There’s still versatility in answering the question “What makes good creative?” There is no one-size-fits-all approach because creative and desired campaign outcomes are different depending on the brand.

To this point, we encourage marketers to play more. Mobile is still a new space and will require continuous testing and playing with how ad creative looks to understand what works and what doesn’t. One of the motivations behind continuously learning and testing is the fact that creative control is in the hands of marketers. As we continue to transform ad creative, this also requires continued measurement and sharing of results with all involved in the process.

You also talked about an in-market analysis in your presentation. How does this approach differ from the retrospective approach?

HannahHannah: As opposed to our retrospective tests, which understand ad creative elements based on known outcomes, in-market tests allow for real-time experimentation and very precise measurement. We can alter specific ad creative elements and understand the impact these changes make on ads. This answers questions like “What happens when I move my logo forward?”

But there is a downside to this approach. Since you’re only changing one ad creative element at a time, it takes a lot of investment and effort to do these studies.

What kind of results did you achieve from these in-market creative optimizations in News Feed?

KumiKumi: For this in-market analysis, we tested video ads. The analysis compared the performance between the same TV ads in two different ways: non-optimized and feed-optimized in News Feed. The optimizations were a mix of techniques selected based on the original TV ads, like adding text or cutting down the spot. Similar to what we saw with static ads, we saw that video ads with lower information loads had a positive impact on Ad Recall.

Two other key findings stood out to us:

  1. Both optimized and non-optimized video ads were successful at moving brand resonance metrics compared to the control groups that saw no ads. For example, optimized video ads drove Ad Recall in six out of seven tests. Similarly, non-optimized video ads drove Ad Recall in five out of seven tests. This tells us that video ads on Facebook are effective at driving Ad Recall.
  2. We also saw positive as well as mixed results when we compared the performance of optimized and non-optimized videos. For Ad Recall, in three out of seven of the tests, we saw optimized video ads outperform at a statistically significant level. In the remaining studies our results were inconclusive.


Our results were not conclusive for other brand resonance metrics, like favorability and message association. We’re still in early days with this research and hope to keep refining our understanding of creative by continuing to test, learn and iterate.

These examples together close our creative loop. By having retrospective and in-market research speak to each other to inspire the creative production process, we can achieve more actionable insights. We hope to continue to prove how, together, these pieces can help avoid campaign waste due to faulty ad creative.

Looking at your collective findings on creative, what do you hope marketers take away from this research? And what do you want them to keep in mind when developing ad creative for News Feed?

KumiKumi: Marketers need to continue working with the fact that creative quality does impact success. By continuing to test and measure ad creative in feed, we can better understand this environment. Optimizing for feed isn’t easy, and we’re all still learning, but ad creative is in our control and together we can work towards better understanding what works best in feed.

HannahHannah: I agree. It’s an exciting time to be in the ad industry. The ways we consume on mobile and in feed-based environments are making us think differently about how to make an impact with ad creative. And this is just the beginning, we’ve started by looking at the question of creative on Facebook, but in reality we know people are being exposed to the same campaign in lot’s of different places. The next questions become how do you optimally plan creative across devices.

But beyond what’s next, right now marketers need to make sure they’re not only playing with their creative assets to see what works for their brand but also measuring these different creative iterations. Applying consistent measurement both retrospectively and in-market can give insight into what’s working and what’s not, as well as help back up your intuitions.

Doing this will allow marketers design more successful ad creative because they know the campaign objectives their ads will be driving. Testing creative, measuring results and using both to inform the creative production closes the creative loop and keeps it rolling.



1 “10 Rules for TV Strategy” by Ipsos, Dec 2014.