Fast Feedback Beats Fast Failure

Summary

In Agile product development, “fail fast” means you release products as quickly and often as you can, so you learn from your failures and evolve a better product. But it’s time for an attitude change. Fast human insight now lets you learn even faster, without the need to fail. It improves your engineering productivity, reduces customer and employee frustration, and increases your odds of getting a hit product on the first try.  

The idea of failing fast has become so central to Agile development that it’s almost a religious belief. It’s been the subject of countless articles and even full books. The idea has been with us for decades. In 2011, the Harvard Business Review called it “Failing by Design,” and according to Google Trends, the phrase “fail fast” has been on the rise since late 2009.

We’ve become so focused on the “fail fast” mantra that it’s easy to forget that the goal of failing fast isn’t actually to fail; it’s to maximize your speed of learning, so you can evolve a better product quickly. Back when cloud computing first appeared, deploying software was so fast and customer research so slow that it made sense to try things and see what worked rather than waiting months to do a research study.

But today the situation has changed. Through human insight products, you can now get customer feedback almost instantly on virtually any idea or experience —within a sprint, without waiting to ship it. This lets you learn even faster than failing fast. The fast-feedback approach gives you several big benefits:

• It saves you time and money because you’re not wasting as many sprints.
• It improves employee and customer satisfaction because they don’t have to go through the disappointment of failing and trying again.
• It improves your marketing efficiency because you’re more likely to have a hit product at first launch.

Here’s how human insight accelerates the learning process in development:

In the discovery phase

This is the time when you listen intently to customer needs and ideas, so you can make high-quality decisions on their behalf. Everyone agrees that it’s a critical part of the product process, but it’s also a notorious time sink. It’s often difficult and slow to recruit the right participants, scheduling is a pain, and there are always no-shows. The interviews themselves are also time-consuming. 

As a result, many companies skimp on the process. Imperfect discovery can give you a false sense of security because the feedback you get isn’t representative of reality:

• It’s common for companies to interview their current customers rather than new ones, because they are easier to find. This puts you at risk of developing for your base rather than growing it.
• Because it’s hard to recruit, companies will often talk to the same people repeatedly, perhaps in an advisory panel. Those panels quickly start to think much like your employees because they have far more context and interaction with you than does your typical customer. 
• Some companies settle for interviewing people who aren’t exactly the target customer but who are more readily available. That produces unknown biases in your learnings.
• We’ve also seen companies focus on social listening (gathering feedback from social media comments) rather than customer interactions, which creates a huge risk that you’ll optimize for the needs of outspoken fanatics and power users rather than typical human beings.

A good human insight solution will let you automatically recruit, schedule, and record interviews with your exact target customer within hours. You should also be able to do “self-interviews” in which the customers read your questions onscreen and answer them without the need for a moderator. This saves you time, and sometimes gets you more candid feedback. Some people will tell their phones things that they would never say to a live human being.

When discovery is done right, you should be able to complete it within days rather than weeks or months. This lets you do discovery for every project, and you can schedule ongoing discovery into free time during other phases. For more information on making discovery interviews easier, see our article here.

In the design phase

(We assume here that your company does a design phase. If you don’t, then this advice applies to the beginning stages of your development process.)

Why do you design before you build? Ultimately, it’s about reducing engineering cost. You want to get the design as perfect as possible before the engineers start putting weeks and months into implementing it. 

The earlier you get feedback on your designs, the sooner you’ll identify problems, and the less rework you’ll need to do later. Despite this, many companies delay customer feedback until the end of design, so they can test a finished high-fidelity interactive prototype. This is actually the worst time to get feedback, because there’s usually no time left to fix any problems you find.

Instead, the best practice is to start gathering customer feedback from the earliest sketches, and again after every significant iteration on the design. A good human insight solution will let you test images and even line drawings, and it’ll be fast enough to get you feedback without slowing down the process. Learn more about adding human insight to your design process in our article here.

During development

There are several situations during development that benefit from fast feedback. The first is when you have to make a significant change to a customer-facing element of the product. For example, it might turn out that an interface widget created by the designer is too difficult to implement and needs to be changed. This sort of change can severely affect the overall customer experience, but you won’t know it unless the change is validated through a quick human insight test.

The second situation is when you’re getting pushback about a feature or other issue from a senior executive. These situations can be difficult to resolve due to interpersonal issues, and can delay a project and damage employee morale. Fast human insight can usually settle the disagreements within hours,  providing objective video evidence of how actual customers react to the issue. We all aspire to operate in a company culture that makes decisions crisply, but when that doesn’t happen, fast human insight is a powerful way to herd the cats. You can learn more about using human insight to align product teams in our article here.

The third situation is pre-launch validation of the first-time experience, especially for mobile apps. This is a situation where failing fast can be especially risky, because bad reviews on the app store can haunt you forever. It’s like a movie; you’re much better off getting it right before you release it. A good human insight solution will let you test the full customer experience on an unreleased app.

After launch

Human insight can make you more efficient by helping you form faster, better hypotheses in two common situations. 

The first situation is when your analytics report a problem with user flow. Typically you’ll see that customers are behaving as expected until they get to a particular point in the experience, and then they’re either going someplace unexpected, or they’re dropping out altogether. For example, picture a checkout process where many customers are abandoning their carts when it’s time to enter their shipping address. Although analytics packages are great for identifying these problems, they don’t do much to diagnose them. Your team may have to run several experiments to figure out what’s wrong. It’s much faster to run a quick human insight test, take people to the problem step, and then ask them to describe what they’re thinking. This will enable your product team to make a high-quality hypothesis on what’s wrong, so they have a good chance of fixing the problem on the first try.

The next situation is improving the efficiency of A/B tests. Depending on who you ask, between 50% and 80% of A/B tests fail to give a statistically significant winner. That can waste a lot of time, especially if you’re testing a part of the app or website that has relatively low traffic, and therefore needs a lot of time for a single test. Using fast human insight, you can pre-qualify the variants for an A/B test in hours, identifying the ones that are most likely to succeed, and flagging any problems in wording or design that might have caused an otherwise good variant to fail. This can significantly increase the success rate of your A/B tests. For more tips on mixing human insight with AB tests, see our article here.

Recommendations

To take your development beyond the “fail fast” treadmill, you should look at using fast human insight in the following situations:

• Recruit and schedule discovery interviews automatically with your specific target customers.
  • Consider using self-interviews to save you even more time (in researcher-speak this is an “unmoderated interview”).
• Validate your designs continuously, from the earliest sketches.
• Revalidate during development whenever you make a significant change to the customer-facing experience.
• Quickly settle disputes with stakeholders by sharing customer feedback videos from human insight sessions.
• Validate your first-time experience before you launch.
• Post-launch, quickly form high-quality hypotheses on how to fix problems surfaced by your analytics.
• Improve the efficiency of your A/B tests by pre-testing and perfecting variants.

The opinions expressed in this publication are those of the authors. They do not necessarily reflect the opinions or views of UserTesting or its affiliates.