Snapchat Ads Campaign Experiments

Campaign experiments are a great tool for you to be able to test different strategies within your advertising account. These have been around for quite a while. The original version is from Google Ads, and if you’re interested in testing those out, we have a video right here that you can look at.

There’s also a functionality for them on Facebook ads. And just like the Google one, we have a video for that, and you can check that out right here. But today, I want to talk about a third platform that’s in the mix, and that’s Snapchat. So in this video, I want to run you through how to set up a Snapchat campaign experiment and talk about some things that might be good for you to test.

I’m in one of our client’s Snapchat accounts because this is going to be the easiest way to show how to run a split test. So every once in a while, we’re going to have to have some things blurred out, but hopefully, you’ll still be able to understand what we’re going for, even with names and images blurred out a little bit.

Now, in an effort to keep the bulk of this video pretty streamlined and cohesive, I’m actually going to go a little bit out of order. I’m going to start by talking about how you can create split tests with existing ad sets and creatives, and then we’ll talk about how to do it from scratch.

So currently, you can see down here at the bottom I’m on the Campaigns tab, and we can’t do anything from here with existing campaigns. But if we go into ad sets, you can now see that there is a grayed-out Run Split Test button over here on the right. So if I choose any of the different ad sets that are listed here, the Run Split Test button will become activated so we can click it, and this will pop up a new window where you can start to create a split test.

You’ll be able to give your campaign a name. The objective will revert back to that of the existing campaign, but then you can choose the test variable that you want to use, which we’ll get into in a little bit. You’d click Create Split Test, and it would send you into the builder that we’re going to walk through here in a little bit. But for right now, I just want to click Cancel.

So now you see what it’s like to create a split test from an ad set. You can also do the same for Ads. Again, you can see the gray-out button on the right. So I would just click the button next to this or a handful of different creatives, and then I would click Run Creative Split Test. And this one’s going to default you to a creative split test because you’re on the Ads tab in the interface.

So the builder that pops up here looks a bit different. Now you can still give it a campaign name, but you get to choose the objective that you want. You’ll then click Create Split Test, and you’ll be added into the builder to get all of the other criteria set up for your new campaign. There is one additional place that you can set up a Creative Split Test from scratch, and that’s going to be in the Creative Manager.

So if I go up to the menu option for Manage Ads and go over to Creative Library, you can see there are a handful of ads in here. So if I just click on one of the ads, I’ll scroll down a little bit, and the preview will show up down here, and I can click Run Creative Split Test from here. So depending on how you want to set up your campaign and if you want to do it from the Ad Set tab, the Ads tab, or the Creative Library, there are a lot of ways you can start down at the Snapchat Split Testing path.

But for now, I want to go all the way back to the beginning. You’ve now seen how you can start split testing for existing assets. Now let’s talk about how you can create it from scratch and go through the remainder of a split test setup in Snapchat. So I’m going to come over here to click Create Campaign, and the first thing you need to do is select your campaign objective.

I’m not going to get into objective objectives today, but if you’re interested in understanding how each of these work, you can check out the video at the top of the screen right now, and we’ll run you through all of the different campaign objectives that you see on the screen. Just to make this as smooth as possible, I’m going to set up as a drive traffic to website or effectively a traffic campaign objective.

Now I’m going to scroll down a bit and in the Campaign Set up section is where we’re going to start the Creative Split Test process. At this point, all we need to do is toggle this ON, and then we’ll be able to see all of the different variables that we can test. With this option, I’m going to go through each of these at a pretty high level and show you how each of these are set up. But here you can see that we have four options for split tests, Creatives, we’re basically testing the ad variants, we have audiences placement and goal strategies.

Let’s go ahead and start with Creative. And as I’m going through each of these different test variable setups, one thing I want you to pay attention to is this builder over here on the left because each of these is going to be the different steps in the process. And as we choose different variables, different setups, and different formats and lists will pop up over here, and there will also be some options of adjusting the test itself that will be in the left-hand bar.

I’ll try to call those out when they pop up. I’m going to click next. Now you can see I’m on the test setup portion of the builder. Here’s where we’re going to fill in all of the details for a creative test. This is going to include the test name, whatever pixels you’re going to use, the budget and schedule, all of those different pieces, the placements, locations, demographics, audiences, devices, everything that we’re trying to target here, even delivery and whatever the goal optimization is.

All of this is going to be the portion of the campaign that is going to be consistent across all creatives. Only the creatives are going to be different. But this setup of audience targeting, ad scheduling is all going to be consistent across the board. But for now, the only piece of this page that I want to call out is going to be this estimated test power portion, which says we.

So if I hover over this icon, it says the split test as it is currently set up has a 24% probability of detecting a winner with high confidence. I can either increase the budget or length of the test or decrease the number of variants tested to improve the test power. Now currently, I only have two creatives in the mix, so lowering that number is not an option.

So that means that I would need to either increase the budget or the length of the test to get this estimated test power to be higher. So if I come over here, scroll all the way back up, just kidding, a little bit lower than all the way back up. I can either increase the budget to be – maybe $1,000 a day instead of $100 a day.

Now, you can see my estimated test power is strong, or if I adjust this a little bit, let’s put that back at 100, and then let’s extend the test and have it run for, I don’t know, 25 days. Now you can see that it’s good. There were different stages in there. We had weak, good and strong. Now, these are obviously just indicators of what Snapchat thinks the test power will be.

So I wouldn’t let this necessarily rule your setup. But it is something to keep in mind that if Snapchat thinks that you have a pretty weak probability of being able to find statistical significance in your test, maybe try and see if you can at least get that. Estimated test power up to good, if not strong, by just making a couple of tweaks, extending the test, increasing budget a little bit and see where you can improve on that front because the last thing we want to do is run a test to determine which portion of our account works best.

Spend all the budget, spend all the time, and at the end of it, still not know what performs best. Okay, so now we want to get into the bulk of a creative split test. I’m going to just click Next because everything in this section is set up. And here’s where we will add in our creative inputs for Creative Number One.

You can see I’m highlighted here, and currently, I have a single image or video ad in place. If I choose to run with this ad format, all of the other subsequent creatives in this test will need to utilize the same ad format. If I change it to a different ad format, you’ll notice that Snapchat tells me that changing the creative type will cause the Snap ad to be deleted.

So, unfortunately, we can’t run a creative test that tests a single image against a story ad against a collection ad. So if you were thinking about doing that, just know that you can’t do that with this testing functionality.

Now, I’m not going to go through all of the different setups for Snapchat ads. If you’re interested in a rundown of how to set these up, you can check out the video at the top of the screen right now. Yes, we do have a lot of previous videos, and yes, I am referring you to them a lot because I want to make sure this video focuses just on the testing portion.

So in this instance, we would get all of the information set up for Creative One. I’m not going to do that. Just to save some time, you could then click Next or click on Creative Number Two in the builder over here. You would then be able to input all of the information for Creative Two. And then, as I mentioned, in the left-hand portion, there’s also some additional functionality where if we want to test more than two creatives, we click add Creative, and then we’ll have a line item for Creative Number Three. We can add up to five creatives in the campaign test.

So you can see here we have creatives One through Five that you could get set up. And you’ll also notice that when I added Creative Number Three, the estimated test power went from good to weak because we did have an additional number of variants in here, and the budget and amount of time that we had is not enough where Snapchat thinks it’s going to have a confident result.

The only last thing I want to mention for structural purposes and understanding how this campaign will be built is that for this current Creative split test, you would have one campaign which will have the campaign names you gave it. But then, each of these creatives will live in its own ad set.

So you can see here we’ll create an ad set for each creative, and each of those ad sets will have all the targeting options, placements, goal optimizations that you had in the test setup portion. Aside from reviewing and publishing, if you have all of your creatives filled out here. That’s the last stage for a creative split test.

So now, let’s keep moving down the list. Hop back to the builder at the beginning. And now, I’m going to change my test objective to audience, and yes, I am willing to lose all of my progress. So now you can see the builder looks a little bit different. On the left, we still have test set up, but then below it we have audience One and two. And then you can see here that ads will be included in all audiences.

So effectively, the structure here is that we still have the campaign, and then each ad set is going to be reflective of each audience. Those ad sets will retain all of the additional information in the test setup. And then, there will only be one ad that you create that will be the same across all of the ad sets that only vary by audience.

So let’s go ahead and click Next. Again, we have all of the different test setup pieces. But if we scroll down these options, you’ll notice that the audience portion is missing. We still have budget and schedule the test details around the pixels, placements, delivery, and that’s it because we want the audiences to be different for each ad set.

Now you can see in the builder off to the left we have audience One highlighted. So in this section, I would give this audience a name that is descriptive of who we’re trying to target. And I adjusted that just so you can see the difference in the line items over here. And then, for audience One, you’ll be able to adjust your location demographics, these specific audiences, whether predefined or custom and the devices that people will use for this Snapchat audience.

So for this option, you can work on trying to test different users in a specific area to see if your campaign performs better with one interest group or another. You can see if a certain custom audience will perform better than something else. Or you could keep the audience members the same but test either the device category, whether Android versus iOS, or you could test if one specific state will perform better than another.

There are a lot of different ways that you can customize the options here. But if you want a rundown of all of the targeting options for Snapchat just to get some ideas of what you could test, we do have another video that will run you through all of those. You can check that out at the top of the screen right now.

Once we have our first audience in place, we could either click Next or head to the audience to make all the adjustments that we want. And again, you can add up to five different audiences for each of these different split tests.

But again, as you add in new audiences, it will make your estimated test power weaker because you’re splitting your budget and focus across more areas, whether for audiences or the creatives or any of the other testing options if you accidentally add up to five of something and you decide you don’t want it, you can come up to these three dots and click Remove.

So it’s pretty easy to get rid of one if you want to see how that impacts your estimated test power. For now, let’s just come over here and click Next. For now, I’m just going to skip all the way down past the audiences to the ad, and again, filling in all the ad details will be exactly the same as it was for the creative type of split test.

But here, you will only have one ad that you put together, so you can choose whichever format you want, add in all the information, and then that one creative will be used across all four of these audiences because the variable we’re trying to test is audience. So to do that and keep all other things equal, we need to use the exact same creative across all the different ad sets.

Now that said, you can add in a new ad here, and then each ad set will include both of these ads. So you can’t customize individual ads for each audience, but you can create multiple ads that will be applied to all audiences. Let’s keep things moving and hop back to the third type of split testing variable all the way back to the beginning of the builder.

And the third option is placement. It reverted all of my settings again. So let’s just go over here, click next. Similar to what happened with the audience after we set up a creative test. Is that now, you’ll see, we’ve got budget and schedule, test details, and now all of the targeting options have come back in location, demographics, audiences, devices, delivery. But now we are missing the placements portion because that’s the variable that we want to test.

So let’s click next. And now we get to choose the placements for the first ad set. Automatic placements is obviously pretty easy, but if we wanted to narrow down to specific placements, we could just click Edit Placement, adjust this ad set settings for content either between content or within.

We can adjust the categories and publishers to our heart’s content. Once we felt like we had the first set of placement targeting options the way that we wanted it, you can then go down to placement set number two and adjust this the way that you want it as well. I know a lot of brands don’t love the idea of automatic placements, and they want to have a lot of control.

But if you’re a little wary of this because you’re not quite sure if you’re going to get good performance out of it, this would be a great way to test setting up an automatic placement versus customized placement campaign just to see if that theory holds true. You would then be able to add in multiple different placements depending on which types of strategies you wanted to run.

Again, you would have up to five, and then you’d be able to create your individual ad creatives or a handful of creatives that will again all be duplicated across each of the different placement strategies that you have. But you will notice that the placement strategy ad testing only limits you to a single image or video or a collection ad. You do not have the other ad formats available.

So depending on what type of ad you’re trying to use, make sure that it is compatible with a placement strategy Split Test. Now we’re going to hop back to the beginning for the fourth and last type of split test campaign and that is going to be down here at the bottom, and that is testing goal strategies.

Let’s go ahead and click next, get to our test setup. And just like we did with the other split test pieces, everything on this tab is going to let you do your audiences, budget, and flight. But then, as we get into the individual ad sets, we can customize the goals that we have. And this is going to focus mostly on the delivery section of the campaign setup.

So if I scroll down a little bit, we’re now focusing just on delivery. And the first thing you do is choose your goal, and then you get to choose the bidding type for that goal specifically. In this account, we were only ever working to drive users to a website. So we only have two goals in place Swipe Up and Page View.

And this is important, and I’ll get to that here in just a little bit. But let’s start off by using Swipe Up. And again, you can adjust your bidding strategies if you want to use auto bid, target cost, or Max Bid to try and get the right performance for the Swipe Up. But let’s say I’m happy with automatic bids. And then, for goal two, I would do the same setup, and then I can choose the Page View goal.

And again, I can customize my bidding strategies for each of the different goals that I have. And now we’re trying to test a Swipe Up versus a Page View. So if I come over here on the left and I add another goal, now I’m in goal number three. You can see here that the goal is diverted to Swipe Up and it’s still set to auto-bid, but that’s already what I have for goal one.

So now maybe I test target cost. Let’s say I want to bid up to a dollar for a target cost, and I feel good about it. I now have three different goals that I’m testing swipe up with automatic bidding, page View with automatic bidding, and a Swipe Up with the target cost. So now I’m going to come and click Next.

And now I have an error. And this is why I wanted to show you what this setup looks like because even though Snapchat let me set up a third ad set, I don’t have three goals in this account to optimize for. You’ll remember when I had the original dropdown, only swipe up and page view were in there? But for this goal strategy split test type, you have to have a different goal.

You cannot only test a different bidding strategy, which in my personal opinion, is a little unfortunate. I think it would be kind of interesting to test auto bid versus target cost versus max bid. But in split testing on Snapchat, you cannot do that. So that might lead you into the question of what goals can you optimize for with a Snapchat goal-setting split test.

And the good news is that split testing is available for all goals. But if you want the estimated test power, you’ll need to choose from the goals listed here at the top of the screen. Swipe Ups, app installs, 2 second video views, 15 seconds video views, story open shares, app purchases, and pixel purchases.

If you have an unsupported goal, it’ll show up as a test power of NA. And any split test campaigns with impressions as the optimization goal will not have a confidence level calculated once the split test is completed. So depending on what your goals are, what you’re trying to optimize for, and if you want to have estimated test power and a confidence level when your test is complete, make sure that you’re utilizing one of these supported goals.

Okay, since I can’t test this third goal, I’m going to just click Remove, move it again, kicked me back to the test setup piece. I don’t want to do that. And for this one, I’m actually going to input something for the creative because I want to show you what it looks like once a test is active. Give me just a second. I’m going to create a super ugly ad and get this ready to go.

Okay, I have a very ugly ad in place. I did adjust some of the test setup pieces just to make sure we don’t spend a ton of budget. And now, I’m going to click review and publish. On this screen is the last chance to make sure you have everything things set up the way you want. Test Summary you can see the audience size, the estimated test power, the goal, the test cells, and the budget for each of the creatives.

As I mentioned, I did adjust this a bit, so I only gave it a $20 total budget, and I only let it run for two days. So this is not going to be a super high confidence test. But another thing I want to call out is this blue box here that says once you publish your split test, you will not be able to edit it. You can pause it at any time.

I believe you can also delete things but the only edits you can make once a split test is live is to change the campaign ad set or add name. You can’t do anything else. So make sure that you scroll through this page. You’ve got the right campaign objective, you have the right creatives, you have the right audiences, and everything is set up the way you want because if you missed something or you set up something incorrectly, you cannot change it without creating an entirely new split test.

So now we’ve had a chance to do last looks; I’m just going to click publish campaign. Let’s head back to the Manage Campaigns tab. Now you can see that I am in the selected campaign, and you will be able to see all of the different performance pieces, all of the different columns, and whatnot at this view. And you can see what is in an active add test because there’s going to be this little formula chemical bottle icon that’s over here, and if your test is still running, that will be white.

But if you have a finalized test, the liquid in this little flask here will be yellow. That’ll tell you that the test is finished and everything is ready for you to make adjustments to it. On the Campaigns tab, you can see that the icon will show up here as well. So it’s easier to see which of your campaigns is currently running some sort of test and which ones are not.

Now, since you saw me set up this entire campaign and there is a bunch of placeholder information, it’s targeting about anybody, and it’s a really ugly creative. Obviously, I’m not going to let this run. So I’m going to turn this OFF because I don’t want to spend this budget. But the last thing I want to call out is if you have your test running and you do find your winner, you can roll out the changes to a new campaign if you want to.

So let’s look at the help section one more time. And in the builder, if your ad set wins and it has the confidence level, you will see this little icon with the ribbon and the star next to it. You can then utilize that winning ad set to create a new campaign, create a new ad set, all of that good stuff to make sure that you are leveraging your top performing variant that you had from your split test.

Personally, I find split testing, whether on Snapchat, Facebook or campaign experiments on Google to be a highly accurate way to test different variants within your account because the platforms do a good job of helping you control for all other variables that could influence a test they run at the same time. They have the same audience. They allow you to only test on the specific piece that you are trying to analyze.

And even though they still might not be exactly perfect. It’s better than we can set up manually at a campaign or ad set level because the platforms will always try to optimize toward what’s performing best, and they will usually throttle the performance of what’s not performing as well, so they don’t get as many opportunities, and then it becomes a self-fulfilling prophecy.

So if you’re interested in trying to determine which of your creatives’ audiences, placement strategies, or goals will perform best for your Snapchat campaigns, I highly encourage you to set up a split test and see what the results are. If you have any additional questions about split testing on Snapchat or split testing across any other platform, please feel free to leave us a note in the comments below.

Written by Michelle Morgan