How to Set Up A/B Testing for Your Online Store (6 A/B Testing Tools Compared)

A/B testing enables online store owners to sift out the elements that are most effective from the ones that require design improvements. It is a scientific approach to maximizing your site’s potential through an optimal design that is effective and directed towards meeting your site’s goals.

When it comes to successful e-commerce, the best of the best online stores constantly A/B test their page elements for the simple reason that there is always room for design improvement. Making an educated guess based on experiments will nearly always prove to be more effective than going down the hard path with trial and error to see what works best with your target market.

In this post, we will cover everything A/B testing – what it is and how it is beneficial for your site or online store. We will also run the rule over some of the best A/B testing tools and plugins that are currently available in the market to help you set up your own system for carrying out the tests.

Let’s get right to it!

What is A/B Testing

A/B testing is a method for measuring two or more versions of a page element to determine which one is more successful based on preset criteria. The core process behind A/B testing is pretty simple when it comes to web design.

For instance, if you have two different shopping carts on your online store, say Shopping Cart A and Shopping Cart B, you can divide your live traffic into two groups and have half of them directed to the page with Shopping Cart A and the other half to Shopping Cart B. After the test duration, you can use a measurement metric to determine which shopping cart variant performed better. Some of the most common measuring metrics for evaluating web design results are:

• Click-through rate.
• Cart abandonment rate.
• Bounce rate.
• Conversion rate.
• Sales or revenue.

From an e-commerce perspective, you can measure just about every page element on your web page or landing page including pricing tables, headlines, promotional offers, copy, layouts, and call to action buttons.

Why You Should A/B Test Your Online Store

The central idea behind A/B testing your online store, or any website for that matter, is that you’re aiming to create the best design possible for your viewership or clientele. If you’re still not convinced though then maybe the following reasons might help you reach a different outlook.

• Boost sales. A/B testing your page’s elements is a surefire way of increasing your sales. Many times the reason customers don’t reach your Checkout page is due to a poor design. By testing out your online store’s navigation or shopping cart placement, you can achieve a design that reflects optimal usability.
• Increase traffic. Have you heard about that really great website that everyone goes to but it has a very bad design? Exactly! No one has. Regularly A/B testing your website’s design and layout is a great way to ensure that your visitors and potential customers aren’t bouncing off because they feel that the site is rather unsightly (pun intended).
• Reduce design iterations. If you’ve ever had to design a site for a client then chances are that you’ve run into some clients that make some pesky demands that don’t really work out all that well from a web design perspective. And this may also force you to take lots of re-dos and re-takes and you end up completely remaking the whole thing. A/B testing reduces design iterations by showing your clients what works and what doesn’t – all based on real-world data.

With all of these benefits that you can very easily reap from A/B testing your website, it would not be wise to forgo it.

6 Tools to Help You Get Started

In this section, we’ll run the rule over some of the best A/B testing plugins for WordPress, Shopify, and other eCommerce platforms that allow you to easily test your website or eCommerce store.

1. Nelio AB Testing (only works with WordPress)

Nelio AB Testing offers the most powerful and versatile conversion optimization for WordPress. The plugin is designed to help you define, manage and keep track of all of your A/B testing experiments. The standout feature on offer with Nelio AB Testing is that it creates heat maps for your website, which allows you to see what your site’s visitors do when they land on your webpage.

The Nelio AB Testing is also fully compatible with the popular WooCommerce plugin, enabling webmasters and online store owners to A/B test their e-commerce stores. In addition to this, Nelio AB Testing is designed as a lightweight plugin that takes all the processing load from your site’s server and swings it back over to Nelio Software’s cloud.

This one-of-a-kind plugin allows users to create alternative content for their page, post, and custom post types. It allows webmasters to go beyond the simple tests and start testing page elements that are more promising. These include headlines, widgets, themes, CSS, and menus.

One of the best things about Nelio AB Testing is that it is incredibly easy to set up for all kinds of users regardless of their technical expertise.

Key benefits:

• Heatmapping
• Fully WooCommerce compatible
• Lightweight plugin
• Cloud processing
• Advanced customization and testing
• Easy to set up

Pricing:

The free version of the plugin features all the basic functionality to start A/B testing your site’s page elements. When your free trial is over you can subscribe to one of the premium plans to continue improving your site’s design.

For $29 per month, you can sign up for Nelio AB Testing’s Basic plan which is great for individuals, bloggers, and small stores. The $89 per month Professional plan is ideal for small business owners and small to medium-sized companies. If you have a large business then you can opt for the $259 per month Enterprise plan.

2. Google Optimize

Google Optimize is Google’s own free website testing tool. It’s natively integrated with Google solutions like Analytics, Ads, and Firebase.

Integration with Analytics gives you access to tons of relevant data that makes it easy to see how the changes you make influence your user behavior. Meanwhile, Google Ads integration allows you to set up and test custom landing page variations for your ad campaigns.

Google Optimize offers a good balance between ease of use and power. There are many advanced features like geographic and user attributes that you can dig into if you’re serious about testing, but you don’t necessarily have to be a developer to set up and run basic tests. 

And because it’s a Google solution, the interface should be familiar to anyone with experience using other Google tools. 

Key benefits:

• Native integration with other Google tools
• Visual editor
• Advanced customization
• Personalization capabilities
• Free version available

Pricing:

You have two options to choose from: Google Optimize (free) and Optimize 360 (paid). 

The free version should be suitable for small- to medium-sized businesses that don’t need anything too technical and are just getting started with A/B testing.

However, if you’re working on a larger enterprise website or need access to advanced experiment capabilities and full multivariate testing, Optimize 360 is the way to go. In this case, you’ll need to contact the sales team for a custom quote.

3. Visual Web Optimizer

Visual Web Optimizer is one of the most flexible multi-variant testing tools available online for A/B testing. Designed with marketing professionals in mind, this tool is used by thousands of leading brands across the globe, including Microsoft, Target, Lenovo, and Toyota. It allows users to create and A/B test different versions of their website, view heat maps, measure visitor segment metrics and target specific user types.

Visual Web Optimizer equally divides your website’s traffic among all the variations and helps you track which one performs the best with your viewership – automatically. With its inbuilt reporting functionality, you know the statistical validity of each variation. The online tool’s point and click system enables webmasters to build campaigns visually within minutes without having to code.

The standout feature on offer with Visual Web Optimizer is that the tool generates an in-depth report that details how your designs performed against the goals you’ve set. In addition to this, you also get a notification every time a particular design variation outperforms the original. Visual Web Optimizer also ships with a heat map tool that enables webmasters to analyze their visitor’s click behavior on their website. This powerful feature highlights the distracting elements of your design and helps you create a more optimized design based on real-world data.

Key benefits:

• Flexible and powerful
• Automation
• In-built, in-depth reporting
• Point-and-click system (no code)
• Heatmapping

Pricing:

The Startup plan is available for $49 per month which caters to 10,000 unique monthly visitors. The Business plan starts at $129 per month catering to 30,000 monthly visitors. Visual Web Optimizer also offers an Enterprise plan starting at $999 per month for large agencies and enterprise deployment.

4. Optimizely

Optimizely is a robust and easy-to-use A/B testing tool that aims to generate user engagement and increase revenue. The tool can be used to experiment with page element designs on any site, including the web, mobile web, iOS, and Android mobile apps. The tool comes with an inbuilt Stats Engine that enables webmasters to take experiment decisions quickly based on statistical data.

Optimizely’s visual editor allows users to brainstorm and create experiments without having to code a single line. The point and click interface is incredibly easy and intuitive. Webmasters can choose to target optimized experiences by splitting their visitors into multiple groups based on ad campaigns, cookies, or geographical location.

The standout feature on offer with Optimizely is that it enables users to segment their test results based on the visitors’ browser, campaign, or device to uncover key insights. This online tool is used by some of the biggest industry names in both the digital and real-world including Netflix, eBay, Sony, Discovery Channel, and The Walt Disney Company, among many others.

Key benefits:

• Advanced segmentation tools
• Ideal for larger enterprises
• In-built Stats Engine
• Visual editor (no code)

Pricing:

The tools are available for a free trial for a period of 30 days after which you can sign up for the Standard, Professional or Premium plan. The Standard plan starts at $999 per month and includes everything you need to start A/B testing your site. The Professional and Enterprise plans are custom plans and will cost you based on your requirements.

5. Convert

Convert is another fantastic A/B testing solution targeted towards SMBs, agencies, and larger enterprises. It offers a fantastic power-to-price ratio with reasonable, flexible pricing plans and all the features you need to test your store or website to enterprise standards.

One thing that makes Convert stand out is the speed. It’s one of the fastest A/B testing tools on the market based on ConversionXL studies. They’ve used SmartInsert™ technology to eliminate blinking that can skew your test results.

Another thing we like about Covert is that it’s fully privacy compliant. The tool uses first-party cookies to track visitor data as default.

Most Convert plans offer unlimited everything (domains, tests, and projects) and come with a bunch of advanced developer options. You can use it to run A/B tests, split tests, multivariate tests, multipage experiments, and more.

If you’re not a technical wizard, don’t worry. You can use Covert’s intuitive, drag-and-drop style Visual Editor to set up simple A/B tests. You can’t do everything in this editor, but you can make basic changes like image swaps and menu reconfigurations.

Convert Integrates with Shopify, WordPress, and Google Analytics, plus all your other favorite ecommerce tools. Their support team is also excellent and is always quick to respond.

Key benefits:

• Great for SMBs
• Blink-free technology
• Visual editor
• Unlimited everything in all plans
• Extensive integration

Pricing:

You can get started with Convert from $699/month with their Kickstart plan. You can also test Convert out for free with a 15-day trial before you buy.

6. Crazy Egg

Crazy Egg is a super beginner-friendly A/B testing tool with a very low barrier to getting started. It’s incredibly affordable, easy to work with, and integrates perfectly with ecommerce stores built on Shopify.

The best thing about Crazy Egg is how simple it is to use. There’s no coding required, no complicated set-up, and you don’t have to be a technical wiz to figure out how to use it. 

All you have to do is select the element you want to test your ideas on, choose your goal, and experiment. You can change the relevant elements on your page super quickly without having to get a developer involved. 

Crazy Egg also allows you to pick the goal you’re hoping to optimize your store for on the intuitive setup wizard and create A/B tests around those goals. For example, you might pick a goal to “Sell More Products” or “Get More Registrations”. 

As soon as the software detects a winning variant in your tests, they’ll send more traffic to that variation automatically. That way, you can maximize your conversions straight away without wasting traffic on a losing variant. 

Aside from the A/B testing tool, Crazy Egg also offers other features that can help you with your conversion rate optimization efforts. For example, Snapshots features help you better understand your customers’ path to purchase, such as Heatmaps, Overlays, Scrollmaps, and Confetti.

Key benefits:

• Easy to set up
• Very beginner-friendly (no code)
• Goal testing
• Heatmapping
• Very affordable

Pricing:

Crazy Egg is very affordable. First, you can take advantage of their generous free 30-day trial. When you’re ready to sign up, plans start at just $24/month.

Wrapping It Up

With the right tools up your sleeve, you can A/B test just about any page element on your WordPress website to increase conversion rates. Whether you decide to go for a WordPress plugin or a professional online tool that is entirely based on your site’s or your business’s requirements.

Were you able to A/B test your websites using the plugins and tools on our list? Do you use a different method, plugin, or tool to A/B test your WordPress site? We’d love to hear all about it so hit us up in the comments below to share your experience!