Facebook IQ Video Value Hero

Making an Impact: Measuring the Value of Facebook Video

People have a growing appetite for creating, posting and interacting with video online, especially on mobile. Facebook, for instance, averages more than 3 billion video views per day—more than 65% of which happen on mobile. With just the swipe of a thumb, people have not only the power to control what content and advertising they view but also the power to decide how much attention they pay to that content. This environment of on-demand video consumption on mobile is compelling marketers to create impactful videos that deliver value. While the paradigm of content consumption has changed, marketers’ objectives have not.

As a step to better understand if video ads are changing people’s brand perceptions and purchase behaviors, the Facebook Marketing Science team commissioned Nielsen to analyze how Facebook video ads move brand metrics (Ad Recall, Brand Awareness and Purchase Consideration) in its BrandEffect database. What we learned is that every part of a video view—from initial impression to a complete view and everything in between—drives value.

We recently spoke with Brad Smallwood, who leads the Facebook Marketing Science team, and Nathan Davis, the Data Scientist who partnered with Nielsen on this study, to understand the methodology behind the analysis and what the results mean for marketers. Edited excerpts of our conversation follow.


Online video content is attracting the attention of advertisers, making it important to understand its impact on marketing. How is Facebook approaching this with measurement?

Brad: The goal of the Facebook Marketing Science team is to help advertisers understand consumer behavior on our platform and measure campaign performance in a scalable and scientific manner. As a company, we’re investing in our video products and we have generally seen good results from well-crafted campaigns with clear objectives. A past study explored what was the best use of video within a campaign and found that a combination of display ads and video ads together best drove business objectives for that advertiser. We can run these types of tests and studies to explore the effectiveness of video ads in driving important metrics like recall, awareness, consideration and, ultimately, sales.

We know from experience that ad delivery technology and changing consumer behavior, like shorter video views because of mobile, can add complexities in measuring performance accurately. We are building solutions like the conversion lift testing product to enable advertisers to measure for themselves what works. These measurement studies are not only helping individual advertisers learn about their campaigns, they are also informing us on ways to evaluate and develop solutions for our clients.

In this newest study with Nielsen, we explore how Facebook video ad formats deliver value. We were excited to release these results at the Advertising Research Foundation (ARF) conference, Re: Think 2015, in March.


In working with Nielsen for this analysis, what was the approach to this research?

Nathan: Our analysis was shaped by questions from our advertisers. One of the primary things they wanted to know was how successful Facebook video ads deliver value and the amount of time it takes to do so. With that scope in mind, Nielsen looked at its database of BrandEffect studies for Facebook video and identified those that included video ads within their marketing campaigns that ran both inside and outside the US, as we believed the results should be applicable globally. In the end, the analysis was comprised of 173 BrandEffect global studies between mid-December 2014 and mid-February 2015. Since we wanted to understand successful campaigns, we then filtered the selected studies for those campaigns that had lift across Ad Recall, Brand Awareness and Purchase Consideration; for example, we used only studies with statistically significant lift in Brand Awareness for the Brand Awareness portion of our analysis.


Are there complexities in measuring video’s impact?

Nathan: Yes, definitely. In the case of this video analysis, it is not possible to randomly assign people to watch different lengths of a video. People choose whether to engage with the content. Typically, in Marketing Science, we rely on randomization to ensure that everything is the same between the test and control groups except the one treatment/stimulus. Then we measure the differences in attitudes (aka lift), which is basically an estimate of how much an ad changed a person’s perception of a brand.

For this analysis, Nielsen started by determining who was in the test group (those exposed to auto-play video impression in their News Feed) and who was in the control groups for the initial BrandEffect studies. From there, we looked at the attitudinal responses for Ad Recall, Brand Awareness and Purchase Consideration. Next, we segmented the test group according to how long each person in that group watched the video, but we also needed a way to estimate how long each person in the control group would have watched the video ad if given the opportunity.

To estimate how long each person in the control group would have watched had they been exposed to the video impression, we used a mixture of propensity matching to leverage the literally hundreds of thousands of variables internal to Facebook and regression modeling. This approach would have been difficult to execute if it wasn’t for the high quality person-level data set that we leverage in a privacy-safe way. With a good matched set, Nielsen was then able to calculate the lift at every second of total time watched.


With this methodological issue accounted for, what were you able to learn from the analysis about video ads impact?

Nathan: Lift is basically a measure of how well an advertiser is able to shift perceptions (Ad Recall, Brand Awareness and Purchase Consideration) through exposure to its ads. But advertisers also want to understand the total impact of their efforts aggregated across an entire campaign. To understand this, one needs to look at the total number of people influenced by the campaign. So while understanding lift is a start, a successful campaign needs to influence people at scale. Once Nielsen calculated the lift curves, our team calculated total campaign impact by multiplying the lift curves and the distribution of people who watched the video for that length of time.


Facebook Video Value

Source: Facebook commissioned Nielsen to conduct a global analysis of 173 BrandEffect studies (Dec 2014–Feb 2015). Test/control design of the studies measured attitudinal impact of an ad campaign.


What the results showed is that most of the total campaign impact happens early in the video, with views 10 seconds or shorter contributing up to 74% campaign impact. That impact comes in the front end of the video, starting even before one second is watched. In this first frame, advertisers have the opportunity to clearly identify their brand and incorporate key messaging—creating lift and impact from the start.


If impact is happening at 10 seconds or less, how should advertisers be thinking about video content?

Brad: Advertisers want to influence people through their ads to care about their brand and buy their product. Understanding how consumers are influenced enables advertisers to craft the best video content. The research suggests that if a brand’s campaign led to 100 people trying a product, 74 of those 100 people will have been reached in the first 10 seconds of that brand’s video ad. Consumers are being influenced from the first frame and more and more from longer impressions. This highlights the importance of maximizing the content and messaging that appears in the first 10 seconds. Ultimately, we believe this is a reflection of how people consume video content on mobile devices rather than something specific to the Facebook video ad product.


Ultimately, as video ads on digital continue to grow in importance and play a bigger role in campaign strategies, what should marketers take away from this research?

Brad: The growth in mobile device adoption has empowered people to choose how much of a campaign they will view. Advertisers can embrace shorter content to take advantage of the nimbleness of the mobile environment and be purposeful in their creative. This will ultimately provide a more meaningful experience for the consumer and diminish waste in an advertiser’s plan.

As with any advertising decisions regarding placement, format and channel, advertisers face a reach/engagement tradeoff and need to experiment to find the right balance given a fixed budget. We see mobile giving brands an opportunity to reach people no matter where they are. As marketers design their digital video strategies, they should consider: 1) focusing on the overall campaign objectives, 2) creating strong creative from the start to make the most impact and 3) evaluating how video works both with and without sound so people are engaged with the ad. Looking forward, we will continue to work with advertisers and the industry to understand how our platform can best be used by advertisers to achieve their business objectives.