How A/B testing can help you prioritise functionality over looks

This blog post is part of our series called “pretty useless”. In this series we explore why having a pretty website isn’t always the best choice for reaching your overall goal – maximising conversion. 

The importance of A/B testing

If you have ever talked to anyone at Umi, you’ll know that we firmly believe that websites are living, breathing things. They deserve regular TLC in the form of audits, updates and refinement. But the key is to not just continuously improve your website – you need to continuously and strategically improve it. There is no point in making random changes, hoping that one of them might result in an increased conversion rate. And if it does – by pure chance – you won’t be able to ever replicate it, as you won’t know what caused the increase.

That’s where strategic A/B testing comes in. Alongside other user testing tools, A/B testing is the most powerful key to website optimisation. After launching your hotel website, ideally using our Growth-Driven Design approach, A/B testing allows you to harness user behaviour and turn it into actual, tangible results. Additionally, it means that you are no longer wasting precious development time on potentially useless website updates.

We cannot stress enough how important it is to allocate some of your time and resources for testing and improving your website. 

But what is A/B testing?

Put simply, A/B testing is a user testing method where you create two variants of an element on your website. Variant A (the original) will be tested against variant B (the amended element). Most elements – such as CTA buttons, colour, text, images, backgrounds, fonts, etc – can be tested. The “successful” variant will be the one with the higher click rate, conversion rate, or any other pre-defined metric. 

However, it only makes sense to test it on a page with enough traffic. Google Analytics can help us determine which pages of your website to pick, but the homepage and rooms pages are usually always a safe bet. Campaign landing pages are also great candidates for A/B testing. Further, you don’t want to test too many elements at the same time, as it can become tricky to determine which change caused the improved results.

Don’t fix it if it isn’t broken

The most important step is to come up with a hypothesis you want to test. This means that you need to have a clear idea in mind before you start setting up the actual test. Again, this is what sets strategic A/B testing apart from just making random changes on your website.

Your A/B testing hypothesis could sound something like this: “Making the ‘book now’ button red will result in more clicks.” This statement doesn’t have to be true but it will allow us to run a test and determine if – in your specific case – it is “true” or “false”. Based on the result, we can then either permanently amend the button colour, or not. Don’t fix it if it isn’t broken.

What you like vs. what your users like

Running A/B tests helps us understand what users like – and that’s not always what we consider ‘pretty’. Often they prefer the simpler, more straightforward or eye-catching option. They might prefer having a button over the main image of your website, as it allows them to navigate to a certain section of your website much faster. Or – and this is the main point I am trying to make – they don’t. We need to observe their behaviour and then respect the choices they make on your website. This way we can make decisions based on actual data and test changes before committing to them. 

Instinct and opinions are important, but where possible they need to be backed up by data. And while we have a great understanding of hospitality websites, every business and its customers are unique and deserve to be treated that way. Therefore, if the result disproves your hypothesis, don’t see this as failure but rather as a learning experience.

Another aspect to keep in mind is that the results of your A/B test might not always align with your brand elements, such as the colour. This fits perfectly with our urge to prioritise function over looks and abandon branding in favour of conversion.

Umi’s approach to A/B testing

There are many ways to approach A/B testing and most will include similar steps. However, our approach puts emphasis on aligning your A/B testing with your overall goals. As mentioned before, your testing needs to be as strategic as possible.

1. Review your marketing goals to make sure the test aligns with your end goal
2. Analyse your website to find suitable elements and pages for testing
3. Create possible hypotheses (e.g. “Changing the colour of the ‘book now’ button from blue to green will increase conversion by X%.”)
4. Prioritise the hypotheses based on impact and effort using the PIE framework (Potential, Importance and Ease). This will give you focus and direction to make sure you are always maximising impact on your end goal.
5. Set up full analytics to make sure you are able to track all relevant metrics
6. Set up A/B test in Google Optimize and define the ‘experiment’ by setting up variants, goals and targeting
7. Run the experiment for roughly 2 weeks, or until we have a 95 percent probability to beat baseline (version A)
8. Review the results to see how the changes performed against your objectives
9. Implement changes where needed and continuously monitor performance

 So what can be tested?

We use a tool called Google Optimize and, in theory, you can test anything and everything. But what does that mean for your hotel or travel website? We compiled a simple list of A/B testing ideas:

• ‘Book now’ button: The two main variables here are ‘text’ and ‘colour’. You can play around with different wordings (e.g. ‘check availability’ or ‘book a room’) or change the button to a more prominent/subtle colour.
• Hero image: Your website’s main image can have a big impact on user behaviour. You can experiment with different visuals, or even test how adding some text and a button would perform.
• Special offers: When it comes to promoting offers, you can test if either a different offer would perform better, or if the current offer might benefit from different wording. You can play around with terms such as ‘free’, ‘complimentary’ or ‘on the house’.
• Landing pages: We love talking about how you can optimise landing pages for Google. However, in this case it’s more about optimising them for users, so they complete the desired steps, such as clicking the booking button or signing up to your newsletter. You can create A/B tests to tweak the length of text, images, forms and your call-to-action.

A few mistakes to avoid

If you follow Umi’s approach, you can’t really go wrong with running your A/B tests. But just to make sure, here’s a few things you want to avoid:

• Don’t just test once, this is a continuous process. The results from one test should form the basis of your next test and allow you to create a new hypothesis.
• Don’t run too many tests at the same time, as you won’t be able to pinpoint what exactly impacted the results.
• Make sure to let the experiment finish before implementing any changes, otherwise you might not collect enough data for the result to be reliable.

Are there any elements on your website that you are unsure about? A/B testing gives you the perfect opportunity to implement them on a small scale before fully committing. Remember to define your hypothesis before getting started and have a clear end goal in mind.