We have to start somewhere to get signals and establish a baseline.
How we doubled interest in a new feature
Litmus, an email QA and testing platform, acquired software that helps you design better emails in the first place which meant we now had to get the word out.
I led the design and strategy to run a series of 6 tests in 12 weeks to double interest in the feature. Here’s a few of the wins (and losses!)
+8%
Retention
+104%
Users creating content
6
Total tests
10 weeks
Timeline
Test 1: Drive users to the new Personalization page with a menu item
Why
Results
Established 4.4% click through rate – which is lower than all other navigation items.
Click through rate
4.4%
Test 2: Add an action button on the homepage
Why
These are our highest performing buttons and our main CTAs – this is now a critical product we want to drive to.
Results
Negligible take rate. It’s clear people are here for testing/building, so we actually may need to pivot and focus on educating users about personalization.
Results
No change
Test 3: Add examples of personalized content you can create
Why
Test confidence: 50/50. This is a big visual change and could potentially negatively affect our other metrics.
Results
Teasing what ‘personalized content’ is and our templates had a net positive effect, without affecting existing metrics.
Users creating content
+104%
Test 4: Repeat a successful pattern where it makes sense
Why
Now that we’ve found that showing a template works, let’s test to see where else it would work.
Results
Variant saw gains ACROSS THE BOARD, showing ~55% improvement in all steps of the LP creation process.
Users publishing content
+55%
Test 5: Give users a reason to convert when they land on the feature page
Why
Because we know showing templates works, we took it a step further by showing industry specific examples on the feature page.
Results
Not only did this immediately improve conversions on the page, it showed us something even more important: HALF(!) of all users who engaged with an example tried the feature and created a piece of content.
Users publishing content
+27%
Learnings + what's next
The faster we’re able to establish baseline metrics and also leave time and space in the sprint to pivot and pitch updates when we have signals from users, the better. Every single day we had to ask ourselves “what are we *really* testing for here, and what’s the best way to get the signals we need before we dive into a “default” experience?”
Think I can be of service to your team? If you’re doing interesting work, I’d love to chat and work with you 👇