Marina Nissim is the business development manager of StartApp (EMEA) and spoke at the App Promotion Summit London 2014 on the topic of using analytics and BI in mobile advertising. The talk covered the following topics:
- Behind The Scenes Of StartApp’s BI Operations
- Dos And Don’ts For Successful A/B Testing
- Segmentation Of Different Dimensions To Test Features
Now we’re able to share the video and audio recordings of the event and you can find this talk and more in our App Promotion Summit London 2014 Bundle.
How To Use Analytics To Improve Mobile Advertising Performance Video:
How To Use Analytics To Improve Mobile Advertising Performance Audio:
How To Use Analytics To Improve Mobile Advertising Performance Transcript:
So hello, everyone. My name is Marina Nissim. I’m the EMEA Business Developer at StartApp. I’m very happy to be here today. Actually I was in the App Promotion Summit last year and I must say it was a great event. And it’s even much more better today to see so many people here. So thank you James and everyone for organizing it. It’s really a great one. So today I’m going to talk about the mobile advertising challenge, how we tackle it, how we manage with the BI operation behind it and how we struggle to find the best products that works for the mobile ecosystem. A few words about StartApp.
To those of you who don’t know our company, StartApp is a mobile advertising platform. We specialize both in monetization, only one, and also in distribution. The company has been doing mobile for the last four years. So we started with mobile and we are still in the mobile area. We have over 250 million active users with overseas billion monthly impressions worldwide. We have global traffic and over 100,000 apps integrated with our SDK directly monetizing their apps on a daily basis. Offices worldwide, San Francisco, New York, China, India and Israel. Some of the companies below you probably recognize are our direct partners for both monetization and distribution. So the vision of the company and where we are heading is mostly the brains, we call it the brains and the beauty of the mobile advertising. What is means that the brains is all about BI business intelligent, collecting the data, showing the right app to the right user at the right time which is quite a big challenge.
And on the other hand we have the beauty side which is all about giving the user the most innovating, appealing ad units so he will also enjoy the experience and the app developer will have a good experience off their app as well. So please take a quick look on the two banners on the screen. The one on the left is pretty blind and bland. It has only three colors and you don’t know where it leads. The one on the right is a branded banner. It’s very colorful and you can recognize the brand behind it. And one is working better than the other. They both lead to the same app. They are both served in the same country. So which one works better? This is a question we ask ourselves on a daily basis and we, when we open the reports every morning when we wake up. It’s important to understand which product work, which product doesn’t work and to get rid of those who doesn’t.
Now let’s add a third element to the equation. This one is, we call it the parachute. It’s a dynamic banner. It reached me basically. And the user wants to click it. The user want to look at it. And this one is also another A/B testing we are doing. In order to see which drives higher motivation in the user. So you’ll be surprised, or not, this one is works much better than the other two. And we develop those kinds of products every two weeks including video ads and native ads and etcetera. Okay, so I’ve prepared for you this periodic table of different segmentations of different type of targeting that we are also doing.
And I will examine only a few, three or four examples to give you an understanding of what the main segmentation we are dealing with on a daily basis. So country, country is pretty basic. We target by country, of course, but we also can’t expect that if one product is very successful in one country it doesn’t necessarily mean it will be a successful in a different market. It probably won’t. And it’s important to understand that and before you’re doing, before you’re testing your products keep the markets you want to test in mind and have all the product tested in the same country to find out what works best. To give you a quick example, we took India and the UK with these two banners and the same banner that works much better in India worked much worse in the UK. Time. Time is very interesting segmentation. In this case it’s not only ticking like it does, it’s also teaching us a great deal of what work and what doesn’t. We don’t compare Monday to Tuesday or Tuesday to Wednesday. We survey Wednesdays. We compare weekends. We do it because we know that each time period, in small resolution, has its meaning.
And for example, I’ll show you here. I don’t know if the graph is big enough but you can see the peaks. The peaks here of the performance. It doesn’t matter if CDR or CR or revenue or whatever. It is showing that Sunday is very strong day for all the metrics by the way. And Monday you have a huge drop. So it’s very smart to compare Sunday to a Sunday two weeks ago to predict what will happen in two weeks. And also it’s important to pay attention to holidays. And, for example, on Christmas the graph of Sunday will be very high. It doesn’t show here but it will be much higher than you see. And on holidays are a great season for ad marketing. Category. Today we have over overall 40 different categories both in iOS and Google Play and it’s a big challenge to find the perfect match between the advertiser category and the publisher category. And we do it. We do it on a daily basis because we understand that some category work better on other products.
For example here, I tested two, we took two apps working with native ads. As you can see one is a music app showing a music ad. And another one is a food app showing a food ad. The yellow one on the left. You’ll be surprised to find out that the music ad doesn’t necessarily work best on the music app. There are other categories like puzzle or productivity that work much better on music apps. And some work much worse. But the intuition is important. Keep it, use your intuition in order to predict what will work but in the end of the day the numbers is what matters. And, you know, we like, we tend to think what will look better, what fits better but when you’re looking on the numbers, the picture. Most of the time it’s much more different than what you predicted. So keep them both. So quick do’s and don’ts for successful BI testing that we are doing with our products and I believe you are doing with your products as well. Okay. So once one size fits all is definitely not the case here. You want your sample population to be as big as possible and in order to test the product. And it’s amazing to think about it. Think in the pharmaceutical industry, different industries, in the drug industry, in the medical industry, it’s enough to have hundreds of test subjects. But in our industry you need hundreds of thousands of test subjects in order to test a product right and get measurable results.
So this one is pretty straight forward. You want your groups to be homogenous as possible. You can’t go compare pregnant women to groups of toddlers. You can’t compare UK users to Korean or Japanese users. They all behave differently. So you need to keep your group homogenous between groups and heterogeneous inside the groups. So, once you split the traffic, when you decide to allocate traffic to different groups, we have, we actually use two main ways to do it. The first one will be by user and the second one will be by traffic. When we allocate traffic by user, we usually care about stickiness, about lifetime value, about user behavior. We care about those kind of things. And what we do, we take specific users and we allocate them different products. Then we see how they react. For example, if you change UI, you want the same user to test it. You don’t want different users to test the same, the same UI. But if you care about performance, or engagement or general metrics, you should test it by traffic like the example with the banner I showed you before. It’s a great example of how we want to increase performance and we split it usually by traffic and not by user. So, as you all probably know, and we spoke about the apps optimization before. The A/B testing is a never ending story and we don’t want it to end. It’s okay that it’s a never ending story. We want to keep testing. Every now and then we have new products. We have new partners on board and we want to give all of them the same attention and have their product tested, increasing performance, increasing quality of the users, obviously. And for us it’s a never ending story. I don’t predict it will end in the near future.
So, thank you very much. Hope you enjoyed it.