Split Testing for eSellers
Editor’s note: This is a guest post by Kim Kohatsu, Director of Marketing at PickFu
With almost 500 million products on Amazon, it’s hard to stand out from the crowd. Split testing is one of the most effective means to optimize your listings and increase engagement and sales on your products.
What is Split Testing?
If you’re new to split testing (also called A/B testing), it’s essentially an experiment between two or more variations to see which variation in the set performs best. These variations might be different product names, featured photos, or product descriptions.
A traditional split test directs a portion of your traffic to one test variation and another portion of your traffic to a different variation. Did one set of visitors stay on your listing for longer? Did one set convert at a higher rate?
How to Run A/B Tests
On your eCommerce website, you can split traffic to all your variations simultaneously. You can write your code to direct traffic to the correct pages, or you can use landing page software such as Optimizely or Instapage, among others.
Another method is to split test using pay-per-click ads. You can run an A/B test in Facebook Ads or Google Ads so that clicks go to different destination URLs. If you use this method, use identical ad creative to eliminate the ad content as a variable. Make sure your campaign is set to rotate the creative evenly; otherwise, the algorithms will favor one ad and serve it up the most.
On a marketplace like Amazon, splitting traffic is trickier to do. You’ll need to rotate the variants over time and compare the results. You might, for example, test version A for the first three days of the week. Then at midnight on the third day, you update your listing to version B and monitor it for the next three days of the week. Of course, this example reveals a limitation of this method: shoppers may behave differently on different days of the week. You’ll need to take factors like this into account when you analyze your results. Other factors might include inventory levels, sales or promotions, and traffic levels.
If you don’t want to alternate your variations manually, you can use Amazon A/B testing software to automate the rotation. Each program has its method of rotating and determining a winner, so try to understand how the algorithms are calculated before you begin testing.
What to Split Test
For eSellers, the main elements to split test are:
- Product titles
- Featured photos
- Product descriptions
Using live tests, you can also experiment with back-end keywords and pricing strategies to see whether they affect sales or rankings.
Limitations of Traditional A/B Tests
We already discussed one limitation of traditional split tests, which is that shoppers might behave differently on different days of the week. But there are other shortcomings as well.
- To gather enough data to make an informed decision, you first need a significant amount of traffic.
- Usually, this means you’ll need to pay to promote your products, which can become expensive.
- Depending on your traffic levels, tests may take days or even weeks. Often, the tests are inconclusive.
- And finally, because you’re using live traffic for your test, you risk an adverse effect on sales or ranking should one of your test variations perform worse than your current listing.
This is not to say that traditional A/B tests don’t have their value. They do. However, they are not the only means to garner valuable insights and improvements on your eCommerce listings.
An Alternative to Traditional Split Tests
PickFu is instant polling software that enables you to quickly test elements of your listing with audiences that match your customer profile and target demographics. On PickFu, it takes less than a minute to ask a market research question, choose the kind of people who you’d like to respond to your question, and PickFu brings 50, 100, even 500 respondents to your poll. Think of it as an online focus group. PickFu test examples include:
Product Name Testing
When one e-seller was developing a gardening product for apartment dwellers, two product names were tested using PickFu: PatioHarvest and PorchFarmer. When the results came in, the difference between the words patio and porch emerged as a major factor in why 76% of respondents preferred PatioHarvest.
In addition, the word farmer didn’t resonate with the audience. Optimizing product names this way helps you understand what shoppers will be more likely to buy. Polling also helps uncover problems you might not have realized were there, such as unintended word associations, difficulty in pronunciation, and unflattering brand perceptions.
Product Photo Testing
One eSeller, Bumblebee Linens, asked 50 female respondents which picture of linen napkins they preferred. In less than 20 minutes, the results showed a new photo was preferred by a 3 to 1 margin over the old one. When the listing was updated with the winning photo, sales of the item improved 209%.
Product Description Testing
In a PickFu poll for a spherical ice tray, 50 Amazon Prime members were asked which product description they preferred. The two descriptions were similar in length and used a bulleted format to lay out the features. But one version used capital letters to introduce each bullet point, such as “PERFECT FOR THE WHISKEY DRINKER” and “PRESERVES THE FLAVOR OF YOUR DRINK.” Respondents found this description easier to skim and more attention-grabbing.
The winning description can now be seen on Amazon, where the product has earned the coveted “Amazon’s Choice” distinction.
Continue Optimizing
You can’t simply list your product and expect sales to come in. By continually optimizing your listings, you’ll help improve your sales, rankings, and seller reputation.
Traditional A/B tests are valuable but have limitations–they can be time-consuming, expensive and too often are inconclusive. However, traditional split tests are valuable for subtle changes that humans aren’t good at discerning (for example, seeing whether changing the order of color, size, and fabric in your product title affects the result).
For many tests, using PickFu is a fast and simple way to optimize product names, featured photos, and product descriptions.
Kim Kohatsu is the Director of Marketing at PickFu, an instant polling service that Amazon sellers use to optimize product listings. With PickFu, eSellers can quickly test product photos, description copy and UI layouts with shoppers who reflect their target demographic, including Amazon Prime members, those within a certain income bracket, or people based on traits such as homeownership, marital status, or exercise habits.