The Lean Startup approach to product development at MagicBus
MagicBus is one of Mobitas’s portfolio companies in the long-distance ridesharing space, founded by Chris Upjohn, Jason Kraft, and Ray Wu in 2016. Since then, the company has gone through the Y-Combinator accelerator program, made a significant pivot from asset-heavy to asset-light, and built a product to modernize and evolve vanpooling, an industry that currently has zero tech investment.
The mission of MagicBus is to transform the way people commute and take cars off the road by making it radically easier for anyone to start, join, and organize vanpools.
I joined the team last year and something thing I immediately learned is to obsess over the principles of the Lean Startup, a book authored by Eric Ries, one of MagicBus’s early investors. Lean Startup applies the ideas from lean manufacturing, a methodology that focuses on minimizing waste within manufacturing systems while maximizing productivity, where waste is seen as anything that customers do not believe adds value and are not willing to pay for.
In the startup world, companies begin with an idea, build a product, raise money, spend money, and release the product to the market. Sadly, many find that the product simply doesn’t garner enough market interest, and funding dries up and they close the shop. Lean Startup is a set of principles for helping startups establish product-market-fit using the least amount of resources possible. One of the most non-obvious but also breakthrough moments for me when I joined MagicBus was realizing that pretty much everything I had learned about operating a business was in large multinational and multi-billion revenue companies. And for these companies, 5-year roadmaps, business plans, SWOT analysis, standard operating procedures (SOPs), are all great and essential. But for a startup, these activities are highly unreliable and wasteful.
Instead, here at MagicBus, we practice the fundamentals from the Lean Startup religiously. This comprises of the following activities:
Talk to customers
Develop hypothesis
Build & test a minimum viable product (MVP)
Validate or reject hypothesis
Repeat
Combined together, in real practice, this means that instead of making assumptions—what the users are looking for, how the design should look, how to describe the value proposition—we apply a scientific approach to validate the ideas through talking to customers, presenting them an MVP, and measuring impact. We eliminate all other activities that do not support us in achieving validated learnings.
***
Talk to customers - As a Y-Combinator startup, we’ve been ingrained with the teaching of talking to customers. Aside from coding, this is the most critical thing that startups must do. Code, and talk to customers. The goal of a great customer interview is to extract data from the customer that will improve your product or positioning. There’s a bad way and a better way to do this.
The bad way: Talk about our features and ask leading questions, “if we build this feature, would you buy it?” After all, you’ll often hear answers that you want to hear because people want to be nice, but what people say does not always translate to what they will do with their time and wallet. In the end, you may fall into the If You Build It, They Will Come trap.
The better way: Talk about the specifics of how the users currently do things (observing them taking such action is even better). Learn about the path that led them to arrive at that problem. Learn about their motivations. Learn why the current way of doing things is difficult. Keep the questions more open-ended so you get to hear what’s top of mind for them.
“Can you tell me about your vanpool program?”
“What are your top challenges? Why is it challenging?”
“Can you elaborate more on what you said about this task being cumbersome?”
The things to extract from these customer interviews are: Who are our potential users? What kinds of problems do they have that we can help solve? Is it just a minor nuisance or is it a burning platform — Is this problem worth solving?
***
Develop hypothesis - After understanding customer pain points, we take this feedback and generate key insights on what are the top problems worth solving for and ideate on potential solutions, whether that’s improving upon an existing feature or building a new feature. The key insight should summarize into a testable hypothesis, for example:
[A certain persona] exists, and they [have a certain problem]. If we [offer a specific value proposition], then we’ll observe [a certain customer behavior and associated metric to change].
Like a good statistical test or a biology experiment, the hypothesis should be falsifiable. A bad hypothesis might look like this:
Adding a feature to book multiple trips will increase user engagement.
This is not a testable hypothesis because it’s not specific enough and any engagement increase above zero is considered successful. Also, how exactly is user engagement defined? Instead, a better hypothesis may be:
Users who ride on the same route more than 3 times per week are frustrated with having to multiple book trips individually. If we provide them the ability to book multiple trips at a time, then we will observe that the average number of trips taken per week will increase by 20% after 1 month.
***
Build and test MVP - After developing the above hypothesis, it’s time to get something into the hands of customers. The key here is to build and deploy an MVP, which Eric Ries defines as:
The minimum viable product is that version of a new product a team uses to collect the maximum amount of validated learning about customers with the least effort.
So, instead of taking 4+ months to design and build a perfect, beautiful, thorough, high-performing, automated, and integrated solution, figure out how you can collect the data needed to validate or reject the hypothesis with the least amount of time and money. Maybe the feature can be enabled simply by texting or emailing users. Maybe it’s a quick frontend update and the backend is humans processing information and coordinating actions (aka the Wizard of Oz approach). Maybe it’s a “coming soon” button to see how many people click on it. Whatever it is, figure out a way to spend 10% of the effort to give users 90% of what you hypothesize they need. (Read more about the 90/10 solution along with other YC’s Essential Startup Advice here.)
I like how this image explains what is and what is not an MVP:
At MagicBus, we are always thinking of how to test new features in a way that requires low development efforts. As an example, we had a hypothesis that B2B customers who operate vanpool programs experience pain points with collecting and reporting ridership data and that if we offered a data dashboard and reporting feature, we would see higher B2B customer conversion rates. Instead of having our engineering team develop a “full-blown” web-based dashboard, the operations team built a dashboard using Google Data Studios in 3 weeks, achieving solving 90% of the key pain points with 10% of the effort.
***
Validate or reject the hypothesis - Finally, after getting the feature out there to the hands of an experimental group of customers, we gather quantitative and qualitative data to either validate or reject the original hypothesis. The results of the MVP experimentation will generally provide guidance to whether we should:
Continue investing in the feature and make it more automated, beautiful, integrated, or
Scrap the idea, or
Pivot the feature based on learnings from the experiment.
For hypothesis around the B2B data dashboard, we measured a 30% higher B2B conversion rate when we present the dashboard versus when the dashboard was left out. Our sales team also gathered quantitative insights that the prospective customers in Zoom video calls were “wowed” when they see the dashboard, and that they were asking detailed questions on how they would use the product — how do I export, how can I drill down, how is a certain metric calculated.
From this experiment and the quantitative and qualitative results, we validated our hypothesis around the B2B customers’ data pain point and how our product can help them solve that pain point. We’ve made the dashboard a permanent feature and will continue to invest in the feature, including migrating from a Google Data Studio-based UI to a web-based UI hosted by MagicBus.
***
Repeat - And then… the loop of developing a hypothesis and experimenting repeats itself over and over again 🔁.
***
Even though it’s called Lean Startup, the approach is not just for startups. There are many examples of how multi-national corporations, governments, non-profits, etc use the Lean Startup approach to drive growth and innovation. For example, the General Services Administration (GSA) established 18F, a consultancy within GSA that helps other federal agencies deliver digital services. When tasked with developing a search service to help small-business owners find federal contracting opportunities, 18F’s designers and engineers created a functioning MVP within 29 minutes! Traditional government technology development programs would have taken months, and 18F was able to get a prototype into the hands of small-business owners quickly and learn how they should prioritize, quickly.
For MagicBus, the Lean Startup approach has guided us to getting desired features to the hands of customers faster, and we have learned where to steer, when to pivot, and when to persist. What is your experience with this approach? Reach out or leave a comment below.