“We had this excel of assumptions of how much it would cost to acquire a user, conversion rates, and the average order size. As we started validating our assumptions, one after the other they all collapsed,” said Yoni Elbaz (Zell 11 – 2012) of Loox.
In his book, The Four Steps to the Epiphany, author Steve Blank72 coined the startup entrepreneurship adage “Get Out of the Building” to entice young entrepreneurs to leave online research and dreamy assumptions for real-world validation.73 His four-step Customer Development process, first published in 2003, involves validating business plans by going out and meeting real customers in order to gauge product-market fit, or the degree to which a product or service satisfies market demand.74 This chapter covers a few techniques used in the Zell Program and popularized by leading methodology over the years.
The topics covered in this chapter begin with Kissing Frogs, which discusses the mindset needed to be a successful entrepreneur, i.e. an understanding that there will be false attempts along the path to finding the proverbial Prince Charming. Mapping techniques like the Lean Canvas Model75 and Root Cause Technique76 are covered in All Over the Map. Validation through the lens of experimentation is covered in Out of the Building and Into the Lab, and understanding the necessity to experiment on what is really important is covered in The Map Is Not the Territory. The honey trap of building the whole product first is covered in If You Build It… and the alternative Minimal Viable Product is covered in Walk Before You Run and MVP Alternatives.It's not only whether the product offering or service solution solves the problem; it is also about solving the problem in a way that enough customers are willing to pay for the product/service in a big enough market that supports a viable business model. In other words, the product and market angle with the highest propensity to succeed and the lowest financial risk
I have spent the better part of the last 15 years – whether in the Zell Program, other programs I have taught in like the Technische Universitaet Darmstadt, MEET (Middle East Entrepreneurs of Tomorrow), WISe (the Weizmann Institute of Science Entrepreneurship Club), Skola (the 81 Unit alumni entrepreneurship program), or others preaching this methodology – urging my students to “leave the building”. I am humbled to say that at my own startup, Horizen Labs,77 I found this incredibly hard for all of the reasons my students had always complained about either directly or through their actions.
Some of these excuses include: the product is still in development and it's not clear what it will actually be able to do; it's hard to define a market, let alone sell a product when the product doesn't yet exist; or if we get too much traction, we won’t be able to fulfill customer demand.
I had always directed students to consider the value of getting feedback against the risk of not delivering exactly what was offered. For me, the thought of building a pipeline that could not be served seemed problematic in terms of PR in the market, but when I sat down to really think it through, I realized I had finally been able to step into my students’ shoes and really feel the prospect of putting yourself out there. It takes loads of time, it involves many false starts, including getting negative responses, which can be demoralizing, it's scary, and because of its high propensity for failure, it’s not a very inviting prospect. However, without getting to market, everything that co-founders and their advisors, employees, investors, and partners strategize about is an assumption.78 If there is no real product-market fit, or a big enough market beyond a design partner or a few early adopters, it does not matter. The tools for laying out the assumptions are many.
In addition to the Four Steps to the Epiphany, and in particular Steps 1 and 2 for Customer Discovery and Validation, at the Zell Program, for many years we have also referenced Alexander Osterwalder's79 Business Model Canvas,80 a visual chart for plotting out a product’s value proposition through its cost and revenue structures, distribution channels, value proposition, etc. Students also create mind maps, flow charts, and databases.81 As presented in the YesChef Case Study, Steve used a version of the 5 Whys Root Cause Technique82 to define the key questions the company would need to answer in order to validate product-market fit:83
In the first question, it's about the supply-side of the business, as chefs are needed to be able to produce the content and offer the distribution channel to their audiences and fans. The second is a direct question about demand from the target market. The third question addresses the product development angle, assuming Hollywood-grade classes were what the target market wanted. Finally, the last question involves the back-of-the-napkin math, or whether this would ultimately be a profitable business. More on the latter question in the next chapter, entitled CAC Is King.
No matter the tools for assessing the market and mapping it out, there is no way to validate the business without getting out there and engaging with customers through endless experimentation.
Unlike the interviews of the ideation stage, where ideas are validated by getting to know the market, as in the case of Yotam Cohen in the Daisy Case Study referred to above, it's not enough to interview customers to assess product-market fit for the simple reason that customers may not know what they need.84 Also, you may not be able to figure out how to ask them the questions in a way that gives you more than polite affirmation. Even if you conduct numerous interviews, the data may be more anecdotal and subjective than quantifiably valid.
This is not to say that it's not a great way to start. We do urge Zell students to ‘get out of the building’ and speak to prospective customers at the outset. Albeit subjective, it helps students get a feel for their space and their offering. But this technique needs to eventually be substituted for experimentation to enable product-market fit validation.
Experiments of many kinds are necessary in order to collect quantifiable data in order to be able to benchmark, track, revert, and redesign the experiments.
It's not all about quantifiable versus qualitative. Surveys are more quantifiable, but there is a big stretch between what people think about what they want and need, and how they will engage with it when it is presented to them.
Experiment optionality is almost endless. But before deciding the experiment, it is critical to understand what is being tested for. A survey can gauge indication of market size and a landing page of organic demand; a mock or demo version can test user flow, etc. For YesChef for example, after plotting out the Root Questions, Steve set out to build a series of simple experiments to test them.
The first iteration involved a ‘sizzle’, or a short clip taken from some competitor’s footage and used to create the feel for the YesChef vision. The sizzle was shown in interviews or sent to hot leads to help customers visualize the offering. Another experiment to test willingness to pay for YesChef was a landing page with the user flow that invited site visitors to enroll and proceed to a checkout for payment. The site was a demo site, and once they reached the payment option there would be an error message, but the data gathered gave strong indicators of product-market fit, at a low cost, and without having to build the whole product solution before testing it.
Many startups fall into the honey trap of thinking that in order to gaugeuser interest, they need to build the entire product first. Sometimes, there are real limitations to the ability to hack solutions the way that YesChef did, i.e. regulatory limitations. For Parametrix85 for example, there were legal regulations in place inhibiting the company’s ability to offer insurance products to the public without proper licensing.86 [See the Parametrix Case Study]
Usually ,however ,the real limitation is a fear of realizing that the underlying assumptions may have been off .Whether it's losing face to one’s investors ,partners ,employees ,co-founders, or to oneself, it is deemed a failure of sorts, and it's amazing the lengths to which entrepreneurs often go to avoid it.
Sometimes, it's a function of having too much money on hand. When there are resources to build without having to prove product-market fit, the temptation to skip quick and dirty hacks grows.
“It’s a disease that a lot of entrepreneurs have. It’s about wanting to bite [off] more than you can chew and not really understanding your capabilities at a certain point in time. There’s a way to be more pragmatic in assessing capabilities,” said Uri Marchand (Zell 8 – 2009) about Overwolf’s first iteration. Fresh with funding, and armed with coding skills, they attempted to build their full product offering, a game overlay that would allow a gamer to do a host of things simultaneous to playing the game, rather than building one capability and testing it out first. [See the Overwolf Case Study]
In the case of Zebra Medical, it was about having too much data. Zebra Medical co-founders Eyal Gura and Eyal Toledano (Zell 2 – 2003) developed AI algorithms based on x-rays, mammograms, and CT scans to help make medical diagnosis scalable. The company was launched only after obtaining a wealth of digital medical images from the largest Israeli HMO. Once they had all of the data, it was difficult to narrow things down. The temptation to build the whole vision out at once was too big. In hindsight, Eyal contends that the better route would have been to build one algorithm at a time, for one disease at a time, and to choose the one where the ability to sell (in terms of geography, accessibility, business model, or distribution channel) made the most sense. [See the Zebra Case Study]
For Liat Mordechay Hertanu (Zell 1 – 2001) of 24me, the segmentation was clear. She was solving problems for people like her, i.e. busy professionals with kids that needed a tool to help them organize their lives. Once she formulated her own list of desired functionality, she reached out and interviewed 50 friends and acquaintances. The list was long, but Liat knew they could not tackle all of the needs.
Early on in their ideation phase, Liat and Gilad quickly agreed that adding all of the features they imagined would be their long-term game plan. However, in the beginning, they focused solely on a lean Minimum Viable Product (MVP), an app that created personalized tasks automatically and provided users with a way to accomplish them via one-tap completion buttons. The MVP included bill payments, a TaskRabbit87 integration for tasks that the system “recognized” by errand type, and a birthday marketplace for sending gifts automatically or posting greetings on social media that were scheduled directly from 24me. [See the 24me Case Study]
Much has been written on the concept of Minimal Viable Product since it was introduced in Eric Reis’ The Lean Startup.88 His thesis of continuous experimentation involved the idea that before building a full-featured product, the simplest version of a viable product was all that was needed. In other words, it is a version of a product with just enough features to be usable by early customers, who can then provide feedback for future product development.
It is the ‘walk before you run’ version of the product. For 24me, the MVP was not the first version of the product, but the most basic form of it. They had a vision of a one-stop shop, but knew they needed to start with the lowest hanging fruit and the easiest features to test. From the interviews Liat conducted, it became clear that birthday reminders were just that. While the initial feature was essentially a calendar reminder, it iterated over time, layer by layer, into a fully-formed offering that helps people not only remember birthdays, but also purchase and send a meaningful gift on time for the occasion.
Since the idea of the MVP was first developed, there have been a host of iterations, including Minimal Marketable Features, which focus on distinct units of functionality rather than the whole product, and Minimal Marketable Release and Minimal Marketable Product, which focus on the ability to service current user needs with must-have functionality, get insights on the product, and get monetization proof|of success.89
Basically, no matter the acronym, the focus of experimentation is around the following risk factor topics:
The experimentation begins from the early life of a startup and continues throughout the journey. The reality of the startup experience is that validating assumptions, or finding out that your assumptions were wrong, are par for the course. Sometimes they lead to a detour, an iteration, or a full-blown pivot. For Overwolf, the minimum part of the minimum viable product was lost on the team (in their defense, the book The Lean Startup, which introduces the concept, only came out two years later). They were so excited to solve so many problems, they proceeded to load endless functionality onto their not-so-minimum product. It took 18 months to unbundle the features and pivot entirely as a result.
Sometimes the ‘walk before you run’ comes from a client request. Mika Kayt (Zell 12 – 2013) of Outgage91 (under its former name Take & Make92) was busy trying to fulfill orders for her DIY Influencer support business, when one of the company’s investors, AOL Ventures,93 asked for help procuring gifts for an event. Though initially this seemed off course for the company, servicing this need turned out to be an incredible risk-controlled experiment in shifting the business to a B2B direct marketing company, and a monumental pivot for Outgage, which will be explored further in the next chapter.
An alternative mode of MVP may also be a function of the regulatory environment. For Parametrix for example, the fact that their solution was an insurance product meant they could not sell a product without regulatory approval, and could not get regulatory approval before they had a product. Since they were offering a product that essentially did not exist beforehand, they needed to experiment with demand and business model viability, and there were no clear benchmarks by which to gauge their solution.
They solved this circular conundrum in two ways: First, they met with and held focus groups with numerous brokers and potential clients to get the lay of the market environment. By definition, it's hard to quantify the cost of business or IT downtime. So the second part of their validation came by gathering and scraping historical data and working the numbers backwards. If they couldn't get customers to try their product, they would look at historical data and assess the value of the proposed insurance policy in hindsight. Backtesting with numbers from the past and showing the actual cost of a technology service provider’s downtime on business activity proved telling.
They were able to present this data to brokers with whom they had been speaking in order to craft an insurance policy that made sense from an actuarial perspective and could be financially viable. At the same time, the historical data was a compelling selling point to prospective customers in their focus group. This process was long and iterative but yielded a form of insurance policy that they felt they could likely sell. The data helped them tell their story to investors as well, and they managed to raise $3 million dollars right out of the program, including from the ZEP Fund, as a ZEP Fund Vintage 5 company.
