An interesting social experiment.
Here at Square One, we measure everything, and that allows us to do some interesting social experiments. Of course, the experiments are all in the name of improving our business, and optimizing our customers’ experiences!
As we’ve mentioned in a previous post, our referral program has proved to be a very cost effective way of building our customer base. Mass media marketing is really expensive, so we’d much rather line our customer’s pockets than some big media company (no offence Pattison Outdoor).
We started out by offering a $10 account credit to anyone that refers someone to us that buys a policy. This is mainly done through the sharing of a personalized URL. We have been tracking the uptake ever since we introduced this program. As you can see from the blue line on the graph below, nearly 40% of our very early customers have referred other customers to us. It’s not surprising that our most ardent supporters signed on with us within a few months of us opening our doors. We think that can be explained by the fact that many of our early customers were friends, family, and investors – all of which really want us to succeed. (Thanks mom!)
Another reason why the “% of customers to refer” appears to be downward sloping is that the longer a customer has been with us, the more time they’ve had to refer people. The challenge with home insurance is that it generally renews annually, and almost all of our competitors charge fees to cancel “mid-term”. That means we need to catch people at the right time of year. Even if you tell your best friend about this amazing new home insurance company that offers great rates and an innovative product offering (for example ;-)), it might be 9 months before that friend is actually ready to buy.
To try filter out that effect, the purple line in the graph below only looks at referrals made within the first 3 months after a customer purchased a policy from us. There’s still the obvious drop off for customers who didn’t purchase in the first 3 months after we launched. This shows that the friends and family effect is definitely real. Longer term, the trend becomes more constant, with around 10% of customers refer another customer to us within 3 months of having purchased.
So now that we had a baseline established, it was time to do some experimenting! One hypothesis we had was that for some people the referral program was too self-serving. We suspected that some people would feel more comfortable promoting us if they weren’t getting a direct benefit from it. This thought was reinforced by some tweets we saw about the program, where the customer took pains to explain the caveat that they’d be getting a referral fee.
To address this self-serving issue, we partnered up with Chimp, a local online charitable giving site. We gave our customers the option of automatically donating all of their referral credits to charity. In line with our overall philosophy of giving our customers choice, Chimp allows you to direct your donations to any registered Canadian charity. Well we’ve had this option available to customers for around 7 months now, and all of 18 customers have chosen it. So much for that hypothesis!
Some factors likely contributing to the lack of uptake are that we still default to the bill credit, so maybe a lot of people just don’t notice that there are other options, or can’t be bothered to change it. It may also be that not many people already have a Chimp account and they can’t be bothered to interact with yet another website/company. In any case, we’ve recently removed the Chimp option from the recommend us widget on our website. You’re still welcome to choose Chimp for your recommend credits, you’ll just have to ask one of our agents to set that up for you. The screen real estate on our website is just too valuable to leave something that virtually nobody was using.
The next hypothesis we had was that maybe if the referral fee was bigger it would be more of an incentive for customers to act. For example, more people might post their personal URL to Facebook. We thought that maybe if we offered $25 (instead of $10) per referral, customers would be more inclined to refer us. After all, it only takes 4 successful referrals to get $100! Obviously the downside of this test (from our perspective) was that we’re the ones paying out the $100. For this reason, we wanted to be sure that the lift from increasing the referral fee from $10 to $25 was greater than the increased cost of the program.
So, we structured it as an A/B test, whereby half our customer base was offered the new $25 incentive, and half was left with the existing $10 incentive. Over the course of 6 months, we did the same for new customers. Our system would randomly assign the $10 incentive to half and the $25 incentive to the other half. We thought it was important that we run this as an A/B test so that we could isolate the effect of the incentive change. If we just did it for everyone, then compared pre and post increase it’s possible that differences could be explained by other factors, like the way we promoted the program. By doing it as an A/B test we know for certain that all other factors are being held constant (or at least consistently different between the two groups). Well without further ado, here’s the result of that test:
Overall, those who were offered the $25 incentive were slightly more likely to recommend us (6.1% vs 5.9%). The lift really wasn’t statistically significant, nor was it large enough, to warrant the much larger cost to us of providing the offer. So, hypothesis 2 has also been shot down. While customers may not be that interested in donating their referral credits to charity, it’s also not all about the money for them.
Customers are more interested in promoting us because they liked our product, service and price. These customers also want their friends and families to benefit from dealing with us. This is in line with the overwhelming number of amazing reviews we’ve been getting. Clearly when a customer is impressed with a product or service, he or she is inclined to talk about his or her experience with others. There doesn’t even need to be a financial gain for the customer.
So long story short, we’ve recently concluded our A/B test. Existing customers that were offered the $25 referral incentive will continue to receive that benefit, but all new customers are only being offered the $10 credit. Sorry people, you snooze you lose! 😉
Any other tests you think we should try? We’re always open to trying something new (as long as no one gets hurt!).