Testing: 5 Steps To Find The Right Formula For Effective Emails

Follow this easy five-step process to create a solid testing program:

1. Ask a question
2. Do background research
3. Form a hypothesis
4. Experiment and collect data
5. Analyze data and draw conclusions

These five straightforward steps are the difference between driving forward an inflexible messaging strategy and adapting your strategy to address the real-world needs and wants of your subscribers.

Step 1: Question

Being inquisitive is the first step to good testing. What about your email campaigns keeps you up at night? Did the design hit the mark? Was the call to action energizing? Was the lunch hour the right time to send?

Let’s imagine I own a design studio. I send a monthly e-newsletter with tips and tricks for refreshing home décor. The e-newsletter also includes a form for scheduling a one-on-one design consultation. I’d start the testing process by choosing a question to answer. For example, what is the best time of day to send my e-newsletter?

Step 2: Research

Get the lay of the landscape by researching. For example, I might start by exploring email marketing resources to see if they contain studies about optimal sending times. I’d then drill in to see if there are industry-specific studies that apply to my business model. It’s important to (smartly) narrow your question. If the existing research overwhelmingly says that sending at 2:00 a.m. in the morning does not produce great results, it’s likely not a time you need to test.

Step 3: Hypothesize

Develop a hypothesis that makes sense for your business and that when tested, will result in clear data. 

Based on the research and what I know about my audience, I hypothesize that an e-newsletter sent at 10:30 a.m. will result in more conversions (completions of the one-on-one consultation form) than an e-newsletter sent at 3:30 p.m.

Step 4: Experiment

Segment your list and begin testing. Keep these dos and don’ts in mind when building your tests:

Don’t: Send one test message to 50% of your list and the other to the remaining 50%.

Do: Use the 10-10-80 rule: One test goes to 10% of your list and the other to a second 10%. Send the winner of these two messages to the remaining 80%.

Don’t: Test multiple variables at a time unless you have an email marketing solution that supports multivariate testing.

Do: Test one variable at a time.

Don’t: Test one aspect of your campaign once and consider it conclusive.

Do: Test one variable in multiple ways across multiple tests. Only make long-term changes to your campaigns once you’ve conducted comprehensive testing.

Don’t: Be discouraged by tests that yield similar results between your 10% segments.

Do: Recognize that tests with similar results can mean that what you tested wasn’t an important component of your message or that both of your messages performed well.

Let’s say that the 10:30 am message resulted in a 33% conversion rate, and the 3:30 pm message resulted in a 15% conversion rate. I would send my e-newsletter at 10:30 am to the remaining 80% of my list.

Step 5: Analyze

Analyze the results of your tests and draw conclusions that inform your future campaigns.

Let’s return to the design studio example. I would not rule out sending campaigns at 3:30 pm after this single test. I would instead perform successive tests to solidify these data. Once I feel comfortable that I’ve sufficiently tested the variable, I would take what I’ve learned and apply it to my future campaigns.

Types Of Tests

The sky’s really the limit when it comes to testing your campaigns. This is just a small sampling of what you can investigate:

• Design (layout, colors, images, text vs. HTML);
• Frequency (number of times per week, month, year);
• Landing pages;
• Link locations (locations of calls to action);
• Subject lines (from minor wording differences to major differences in approach); and
• Timing (day of week, time of day, holidays).

If I had one piece of advice to give you about improving your email campaign performance, it would be to test often and smartly. That is, don’t test for testing’s sake, but do experiment as long as it makes good business sense.

As Chief Product, Marketing and Strategy Officer, Jeff Revoy is responsible for marketing, product management and corporate strategy. Revoy brings more than 20 years of experience in Internet, search and social media at both early-stage and Fortune 500 companies. He worked for Yahoo! in the U.S. and Europe. Revoy and his team helped turn around the European search business and led Yahoo! Answers to a market leadership position with more than 150 million monthly users. Previously, Revoy served as President and General Manager of Interactive Services at Embarq, a Fortune 250 telecommunication company. In addition, he has held leadership positions with InterTrust and GameHouse.