Product Discovery is an incredible way to:

• Frame the problem
• Interact and learn from your end users
• Conceptualize and prototype solutions
• Test assumptions and concepts

Doing all of the above will help minimize risk in developing the specific features in each sprint, as well as minimize risk to the product’s raison d’etre, or purpose for existence.

It’s a major departure from how products were conceptualized and built just a few years ago, when executives would typically develop the product strategy – often without user research, data, and/or customer input – and task their developers with implementing it. Often leading to many failed products – and companies – as well as wasted time and money for all involved.

However, as many product teams continue to shift to Agile, they still continue to fail in properly executing Product Discovery. Score your team’s performance using the reasons listed below, where product rockstars will be 0/5.

Reason #1: You’re not doing any discovery.

The first – and easiest – way you can tell if you’re doing product discovery wrong is if you’re not doing it at all. Honestly, this should be a crime. Think about it this way: would you bet the total product development budget on a coin flip? Heads I win, tails you lose. That’s the risk you take if you’re not doing any discovery – you might as well just give me your money.

Simply put, you should be performing discovery for all digital products – new or existing – especially if those products are externally facing with clients, customers, or the general public. Discovery provides the critical data you need to make informed product decisions around user segments, features, requirements, etc. You’re flying blind without doing any discovery.

Reason #2: You’re not directly engaging with your end users to gather insight.

Building great products requires a deep knowledge and understanding of the people you are building the product for. The best way to gain those insights is to engage with them directly: talk with them about their common tasks and goals, background and influences, as well as have them provide feedback – capturing their feelings, likes, dislikes, and ideas – on one or more prototypes. The rule of thumb is to test each opportunity with at least 15 end users in order to see a trend in the feedback data.

It’s important that you focus on listening intently to everything your end users say. Take a lot of notes. Record these interactions if possible.

Engaging with them will allow you to empathize with them, where you’ll be able to understand – and document using the Empathy Map above – their needs, thoughts, emotions and motivations. The good news is that you have a wide range of methods at your command for learning more about your end users (see Reason #5), where we recommend using more than one method. Doing so will allow you to build out sophisticated personas and put yourself in their shoes whenever you have to make product decisions.

Reason #3: You’re not doing dual-track discovery AND development.

Many product teams think it’s sufficient to just do discovery once at the beginning and never do it again. Perhaps they think, “We’ve validated our product concept and our users will never change, nor will our competitive landscape, or the tech-stack we used!” I can tell you that story never ends well, just ask AOL. Or MySpace.

Discovery should be a continuous activity for product teams – starting in Sprint 0 – and continuing in every sprint at the same time as development. The Discovery track will provide the validated opportunities, and the Delivery track will build shippable software.

Reason #4: You focus all of your discovery efforts on trying to validate your opportunities.

Discovery will not be successful if the Product Team has the wrong mentality going into the discovery phase, where they think that every opportunity in the backlog is awesome and will be validated.

But isn’t that the purpose of discovery?

Not quite. The actual purpose is to ensure that you’re building the right product before you actually build it. And through this process you separate out the good ideas from the bad… and trust me, there’s a lot of awful ideas that get thrown into the backlog. That’s why you should always be a skeptic during discovery, and actually work hard to invalidate the bad ideas.

So why do product teams try to force the validation of every opportunity?

Partly because they’re misinformed through poor Agile training and inexperienced Agile practitioners. Some teams just rush through discovery. But mostly it’s because product teams are deeply passionate for each and every opportunity in their backlog, because THEY are the ones who thought of them, duh! And the only thing standing in their way of launching that guaranteed awesome feature is that pesky little thing called user validation. And they will do whatever it takes to validate any and all concepts, even if it means talking with fake users (i.e. friends), skewing the results, or even cherry-picking data to make it seem like users are interested (which I witnessed firsthand by executives at various companies I’ve worked with). Anyone with enough data can and will validate any opportunity, but Product teams need to realize that this is just as bad as not doing any discovery, where this will lead to the development of failed features and/or products.

You have to be absolutely certain that they’re giving you honest and valid feedback, and any reasonable doubt now will cause issues later once it’s deployed. It’s like shopping at the mall, if your users don’t love the concept now then they definitely won’t download and use your app later.

Whenever I’m doing discovery work via in-person user intercepts, I typically throw out up to 3 positive responses for every 20 users I talk with. Mainly because some people will rush through the process, or they just want the reward you’ve dangled in front of them (e.g. Starbucks gift card, etc.), or they’re just too nice to be negative. The same concept holds true with data analysis, where you typically eliminate the outliers to normalize the data.

Reason #5: You only use one method for testing your opportunities.

The problem with only using one method is that there’s always some level of uncertainty baked into each method, and relying on one method will significantly increase your risk of the assumptions being improperly validated. Try using two or more methods below for testing each opportunity in your backlog and your odds of success will substantially increase.

• A/B testing
• Analytics – Goal and path analysis
• Beta testing
• Card sorting
• Clickable prototype
• Eye tracking
• Focus groups
• In-person paper prototype testing
• In-person interviews
• Multivariate testing
• Online surveys

So how’d your team do? Add your score in the comment field below.