Usability testing – a sobering experience

Based on all the positive feedback we received about our online quoting tool, we thought that we had done a pretty good job! But one thing we have learned over the years is that it’s always wise to test things out. So, we recently subscribed to usertesting.com so that we could have a few people do a usability test of our site. We’d actually been urged to do this over a year ago, by friend and usability guru Paul Hibbits. Sadly, it took us longer to do it because we thought we had more important things to do than watch people use our site – boy were we wrong!

For those of you not familiar with what a usability test is, the concept is that you get a bunch of people that fit your target user profile (but haven’t used your website before). You give them a goal, then sit back and see how they do. Ideally, you want them to talk about what they’re thinking every step of the way. That way you can see exactly where things are going off the rails. And in our case, did they go off the rails quickly! 🙂

We opted to get 3 different people to each perform a test that takes at most 15 minutes. These users aren’t full-time “professional” testers, but rather average people who are willing to do a test from time-to–time. The users earn a small fee for each test they take. Here’s an example of one of the usability tests done for our site (hopefully you don’t find it as cringe-worthy as we did):

You can choose the characteristics of the testers you want. As the testers proceed, a video of their screen is recorded along with a running audio commentary of what they’re thinking. Pretty simple in concept, and you wouldn’t think it would result in any earth-shattering revelations. From reviewing the results of just those 3 tests, we came up with over 30 changes we wanted to make to our site. Some examples of the changes we’ve recently made:

  • We rewrote the copy for every sentence in the quoting flow, trying to be more succinct and clear.
  • Error and help messages have been moved in-line with the questions, and out of their pretty pop up “bubbles” on the right side of the screen. We thought they looked cool, but users weren’t noticing them. The old approach was frustrating users.
  • We’ve switched from using Google’s autocomplete address entry tool, to a new one offered by Canada Post. We’re now paying 7 cents per lookup (whereas Google’s is free), but Canada Post’s works better for entering residential addresses, and supports apartment unit numbers. We think 7 cents is a small price to pay to make it easier for customers to get a quote.
  • For questions that users were having difficulty answering, we’ve tried our best to find another way of asking that question, recognizing that we still needed to collect that information. Examples of this are asking for the number of bedrooms, instead of square footage, if you live in an apartment, which allows us to come up with a close enough estimate of the size of the home. Another example is that we now ask for how many cars fit in your garage, instead of its square footage.
  • If you’ve answered something incorrectly, we’re now performing real-time validation so you don’t have to submit the form before being notified of an error.

Now that we’ve gone live with the “fixes” from the last round of usability testing, we plan to go through the exercise again. Previously, I wouldn’t have thought this next round would come up with anything much, but now I’m not nearly so confident.

Do you have any feedback on our site? While it’s nice to regularly hear praise, we’re actually surprised we don’t hear more gripes. Clearly, there are gripes, based on those testing results. So this is your invitation – if you find anything confusing or frustrating on our site, please send me an email or call me at 1.855.331.6933 ext 103.

Send an email to Jason





Find the right policy

Own or rent? House or condo? Choose from these policy types:

Get a quote

You can get a quote in less than 15 minutes. Click the button to get started.


Get an online quote now

Learn more

Want to learn more about your home? Check out the various resources centres available to you, and let us know if you'd like a certain topic added.

blog comments powered by Disqus