A/B testing has proven to be one of the best ways to consistently get higher and higher revenues and usage from your website visitors. The last few years have seen an explosion in the amount of businesses running A/B tests on websites, emails and now on mobile too.
A/B testing is especially useful in mobile marketing. Mobile is fairly new, at least when compared to the desktop. That means that there are a lot of assumptions out there that need to be tested to see if they work. It’s fertile ground for experimantation.
Let’s take a look at 5 examples of A/B tests, and the lessons you should learn from them.
Example 1 – Mobile vs Desktop
Vegas.com is the go to website for the city of Las Vegas. Their A/B test compared not having a mobile site versus having a mobile site with content specially suited for mobile.
The results were pretty amazing. They found that even a small mobile site, without all the functionality of the desktop version, was extremely successful. Their traffic went up 16% and their bounce rate went down 22%.
The lesson here for you to learn is twofold. First, A/B testing can be very successful when it comes to comparing desktop usage and the mobile web, since they are easily compared. Second, even if you can’t afford to build a fully functional mobile web page and have to rely on a landing page, having a the most basic mobile site can give a great return.
Example 2 – Tweaking Your Layout
Your mobile site is already very important to your company, everyone knows that. But how can you make it better? That’s the question the owners of the site ApartmentGuide asked. To find out if a new format would generate more leads, they conducted an A/B test between two versions of their mobile site. Their question was: could placing sponsored search results generate more leads and/or more income?
They thought that their leads would go down, but they ended going up by 1.6%. So what’s the lesson you should learn from this A/B test? Again the lesson is twofold. First, you never know what the results of your test will be, it could be what you expect or it could surprise you. Second, even a slight tweak to the way your mobile site works can have a significant impact on traffic and revenue.
Example 3 – Your Important Stuff Goes at the Top
This example isn’t mobile explicit, but works just the same. This test proposed that placing the email or social signup form at the top of the page would increase conversion rates. Their results? A 336% increase in signups over their testing period.
What is most important to your success? That is the question you should ask when it comes to your mobile website. If changing something like moving a signup form or advertisement to to top can have that big an impact, why wouldn’t you try it? On the desktop, there are certain spots on the web page the convert better than others. The same is true on mobile, where the top of the web site is seen more than the bottom. Performing an A/B test is a great way to find out if this is the case for your content.
Example 4- Your App Icon Matters
This article tells the harrowing story of a Tap Tap Tap release where they seriously messed up by not doing an A/B test. If you’re an app developer, you need to test your icons. Which one works best? It can’t only be what looks good to your, or what looks good to your team. The Tap Tap Tap guys found that after correcting for their mishap, their sales skyrocketed up into the top 10 of the app store.
Your icon is like a little banner advertisement. Not only that, but it is the first thing people will see when they are searching through the app store. Whether they should or not, people do judge books by their cover, and they will judge your app by your icon.
So test your icon with an A/B test during a beta launch to determine what icon style best suits your app.
Example 5 – Don’t Be Afraid of Change
RunKeeper recently ran an A/B test to compare two designs. One was their a simple list-based design, another was a totally redesigned app that surfaced other features.
Their goal was to encourage people to use thee app for more than keeping track of running. What they found is that the design that surfaced other activities and increased user engagement of those activities by 235%.
So what should you learn about this A/B test? RunKeeper admitted how risky changing their home screen was. It’s not a small move. A/B testing can really help de-risk a big change like this. Many testing tools will let you show the new layout only to 10% or 20% of users, to start small, measure the impact and then roll it out.
Don’t be afraid to test something that is completely new if the old way wasn’t working.
Your mobile web experience is fluid. In many ways mobile is just as important as the traditional web, and its importance will continue to grow over the next few years as more and more people move to mobile for their primary computing experience.
A/B testing allows you to control that fluidity and make sure the changes you make actually work before presenting them to your entire audience. This not only provides you with the opportunity to avoid costly mishaps, but also allows you to test a lot of options quickly to see what works best.
So what are your experiences with A/B testing? What’s the biggest item you want to test next? Leave your thoughts in the comment section below!