10 Years Is A Long Time: The Difficulty of Predicting Interesting Markets for Startups

Entrepreneurs and startup investors make bets on the future. That’s the nature of building startups. In order to do that well, you need to make correct predictions about which markets (and specific startup ideas) are going to be successful. It’s well-known that that’s difficult, but how hard is it really? A decade-old list of startup ideas by one of the most prolific entrepreneurial experts provides an interesting perspective.

While cleaning out an old stash of digital notes (something I highly recommend for inspiration purposes) I found a blog post that is almost exactly 10 years old. It was written in July 2008 by YCombinator founder Paul Graham and listed 30 areas in which the famous startup accelerator was seeking to invest. The list was meant as an inspiration for aspiring startup founders to pick an interesting market for their project. It’s a remarkable, very concrete snapshot of startup investor thinking at the time. And with the benefit of hindsight, it’s a fascinating illustration of how difficult it is even for startup gurus with an incredible breadth of experience to predict which markets will be relevant in the future.

10 years is a very long time in technology. For context, when this list was published, the iPhone had just been on the market for about a year, and Apple’s App Store had just been launched days earlier. So it’s not too surprising that the disruption through mobile apps is not yet a theme – the word “mobile” is mentioned exactly once in the whole post. AWS was just about two years old and just bearable for the earliest of adopters, so the future of cloud computing was still unclear. Social networking was still a relatively minor industry, with Facebook just having about 100 million users at the time and Twitter just about a year old.

A decade is also a very relevant timespan in the startup world because it’s the default lifetime of most venture capital funds. VCs are trying to make investment decisions early in their funds’ 10-year lifecycle that should pan out within 5-10 years, so being able to pick interesting markets (and of course the startups the VCs hope will be successful in those markets) is key for a VC firm’s success. And, as we will see in this example, it’s incredibly hard, which is of course the reason why venture capital as an asset class rarely performs better than the stock market.

A decade is also a time horizon startup founders should carefully consider for their own life plans. An old word of wisdom in startup circles says that if you’re not willing to deal with a particular topic for at least 10 years, you really shouldn’t start a company in that field. Most startups take way longer than you think to develop, so picking a promising market (and one you like as a founder) is essential. Of course, most startups pivot during their lifetime, many even several times. But it’s rare that a startup can survive a transition into a very different market. Founders will tweak their products, target segments and business models, but they will typically stay in the same general market, so choose wisely.

To compare YCombinators’s predictions with what has really happened over the past ten years, I took a list of current and recently exited unicorn startups with a valuation of over $1 billion as a proxy for successful markets for startups. That’s of course incomplete, but since VCs are typically shooting for these kinds of outsized hits (and most founders wouldn’t mind having a billion-dollar startup either) it’s probably as good a list as any. I also took the liberty of simplifying and modernizing some of the terms of the original blog post to make it easier to digest for an audience in 2018. Once again, this comparison is not meant to grade YCombinator’s predictions, it’s just a great example to illustrate the overall point.

The details are listed at the end of this post. But as an overview, out of 30 market ideas, just 11 panned out about as expected, producing large, successful startups (not nearly all of them through YCombinator of course). Four of those ideas are closely related to the overall trend towards SaaS and cloud computing, which, while early, was already very clearly visible in 2008. Not surprisingly, extrapolating current trends is typically easiest. But there were also topics that were still nascent in 2008 that turned into major waves, such as online learning.

A next category are market ideas that found some success, but in different ways than expected. This includes categories that are now dominated by large corporations (such as web-based office apps, which are fully controlled by Google and Microsoft), as well as topics that saw some progress without real breakthroughs (new payment methods are an example). 12 of the 30 ideas can be categorized as somewhat successful, but implemented differently than expected.

The third category are ideas that have yet to come to fruition. 7 ideas fell into this bucket. We are all still waiting for somebody who solves the problem of e-mail overload or fixes digital advertising.

Finally, I added another category of 7 markets that were not on the list of 30 ideas, but turned out to have a very major impact on the past decade in startup creation. This includes ride sharing services, direct-to-consumer startups and messaging technology. Who would have predicted that a simple messaging solution like WhatsApp could be a $19 billion exit? This list could be extended with many other niches, but these seemed to be the most important ones.

Again, this list is of course only one view of the 2008 market of ideas, but it’s commendable that YCombinator was so transparent about its thinking, while most VCs only publish a very vague “investment thesis”, if anything at all.

There are a few important lessons that can be learned from this analysis:

You will get the majority of your predictions wrong. Most likely, this was one of the most competent and thoughtful predictions of startup markets at the time, and still only about a third turned out as expected. That’s actually pretty good. Most people will not hit more than 20%.
The safest markets are of course those that already see a lot of traction and have produced some relevant companies, such as SaaS. Salesforce went public in 2004, so in 2008 it was clear that SaaS was probably here to stay. Filling one of the countless niches that such a foundational technology opens up is a good way to build a successful company. Downside: Obvious markets attract a lot of competition. As a datapoint, I analyzed 108 SaaS startups in 2007, and most of the smaller ones have gone out of business.
Major technological waves that can boost or create several markets are often hard to see. Back in 2008 it was not clear at all that the smartphone would change our lives so thoroughly and enable tons of disruptive startups. These broad platform technologies often make things possible that simple couldn’t exist before. Uber and Lyft are great examples. Mobile gaming is another one.
Hard problems are sometimes really hard, even if everybody wants a solution. E-mail, video conferencing and spreadsheets were already awful in 2008, and they still are.
Incumbents coming into your market is a problem, but not by far the greatest one for startups. Typical investor question: “What happens if Google enters your market?” Correct answer: There are probably more important problems. OK, if you were planning to replace Microsoft Office with web-based software in 2008, you were out of luck. Google did that, and Microsoft followed suit. But there are plenty of examples for segments with strong incumbents where new companies found attractive, substantial niches. The key seems to be to find topics that are not a high priority for incumbents, and most likely they will leave you alone.
Blind spots often arise in “boring” sectors. Selling razors online directly to consumers doesn’t exactly sound like the stuff of entrepreneurial dreams, and yet it produced a unicorn. So did food delivery services and e-commerce sites for furniture or baby clothes.

Bottom line: Reviewing your own or other people’s predictions after a while is a very healthy exercise because it helps you understand biases and blind spots in predictions.

————

Here are the details per category:

Correct predictions

Here are the fields that YCombinator found interesting and where we have seen successful startups:

Music and movie distribution: Spotify, Netflix, Hulu, Roku, YouTube and others without a doubt have revolutionized how we consume media. Streaming has removed many of the copyright issues that plagued the industry in the early 2000s (think Napster), so this was spot-on. Having said that, the big loser was the record industry.
Outsourced IT: Thanks to cloud computing and simple SaaS solutions, no small company nowadays really needs a dedicated IT department with its own server hardware. More and more large corporations are moving to the cloud as well, getting rid of much of their legacy infrastructure. Dropbox, which plays in this sector, is famously one of YCombinator’s biggest hits, so this prediction was spot on. However, many of the gains flowed to established companies like Google, Microsoft, Amazon and Salesforce.
Enterprise software 2.0: As mentioned in the previous point, SaaS in the past decade has developed from a somewhat exotic category to the main way to sell and use software, so this one was correct too.
More variants of CRM: Salesforce, which was already well established in 2008, now dominates the core CRM market, but there are plenty of more specialized companies such as Hubspot, Intercom or Drift that tap into more narrow CRM-related use cases successfully.
Dating: Without a doubt, Tinder and other dating apps have significantly changed how people meet. Spot on.
Photo/video sharing services: Instagram, Pinterest and Snapchat are unicorns that are less then ten years old. There are several more in other markets, particularly China.
Online learning: Ed tech is a big and rapidly growing sector, producing even a few unicorns such as Udacity or Lynda.com (and several in the Chinese market). Also, most established universities have started to invest heavily in online learning, so this was a clear hit.
Application and/or data hosting: A clear yes. Cloud computing is ubiquitous and goes well beyond pure basic infrastructure. Startups like Dropbox, Box, GitHub or Snowflake benefited hugely from this trend.
Finance software for individuals and small businesses: Personal finance has been a very successful category with notable players such as Betterment, Robinhood or Lending Club.
Hardware/software hybrids: Another broad category, and you could say that in the age of the Internet of Things countless consumer products are such hybrids. Universal hardware platforms (such as smartphones) are safely controlled by huge corporations, but startups in smaller niches pop up all the time – including some real breakouts, including Nest, Roku, GoPro and Ring. But building hardware in combination with sophisticated software is still hard, as demonstrated by an endless stream of failed Kickstarter projects in this realm.
Startups for startups: “Startup accelerators” was a forbidden concept for a long time, but in past years they have sprung up left and right – although typically not necessarily as profit-driven businesses. But there are plenty of startups that sell the proverbial shovels for the goldrush, mostly notably mega-landlord WeWork.

Sort of right, but implemented in a different way

There were topics in the list that have manifested themselves as significant markets, but in a quite different way than anticipated.

Simplified browsing: The experts called for better web browsers, and what we got were mobile apps. Right problem, different solution.
New news: YCombinator correctly diagnosed that the news industry was going to be disrupted, but probably nobody expected a phenomenon such as the Facebook news feed. However, there have been some unicorn news startups such as Vice and Vox, but they’re going through their own disruptions currently.
Something your company needs that doesn’t exist: That’s of course a very fuzzy definition, but there have been some B2B startups that solved annoying and less-than-obvious problems with great success. Zenefits, GitHub or SurveyMonkey are good examples.
Web Office apps: Porting productivity apps to the web has been a successful wave, but the main beneficiaries are Google and incumbent Microsoft, with very few startups having made a mark. There is a new wave of productivity startups such as Coda, but it’s still early days.
Tools for measurement: Kind of. The intention of this point were solutions to measure operational data such as employee productivity, but what we got were a bunch of general-purpose data analytics startups such as Domo. Measuring the specifics of a business is still complicated.
Off the shelf security: While there are a bunch of successful B2B security companies, this point referred to home security. And indeed, that’s a rapidly growing sector as part of the overall smart home wave. Smart lock maker Ring recently was acquired by Amazon for $1B, and that’s a typical symptom of large players moving into this sector before startups were able to get very big.
New payment methods: Yes and no. Obviously there are cryptocurrencies that use a truly revolutionary approach to payments, but they’re still far from mainstream adoption. However, there have been several huge startup successes in different variations of solving particular payment use cases, most notably Square, Stripe, Transferwise and Adyen.
The WebOS: Google launched Chrome OS in 2011 and has managed to establish the browser-based OS as a credible alternative to Windows and MacOS in some market segments. But that sucked the air out of the segment for startups.
Shopping guides: This idea never really made it as a standalone category, but you could argue that every major e-commerce site now has its own recommendations, so in a way it did happen. There are of course specialized vertical players for certain kinds of products, but nothing at tremendous scale. Stitch Fix is a notable exception.
A buffer against bad customer service: This is a very broad topic, and definitely one that was a key influence for many successful unicorns. You could argue, for instance, that Uber is really a solution for bad customer service in taxis, but that’s of course only a part of the overall value proposition. One thing is clear though: E-commerce and sharing economy startups (along with giants like Amazon) have clearly raised the bar on what expectations for good customer service are.
A Craigslist competitor: There’s no new general-purpose classifieds platform that went up directly against Craigslist, but instead we’ve seen a proliferation of more specialized marketplaces that raised the bar on quality. Another one of YCombinator’s biggest success stories is Airnbnb, which did this for vacation and business apartment rentals. Rumor has it that Airnbnb initially scraped Craigslist rental ads to fill its platforms, so there was clearly an element of replacement and upmarket disruption at work.
Easy site builders for specific markets: A few companies such as Squarespace have built impressive businesses with website builder software, but they tend to be fairly horizontal in their scope. Turns out that breaking down a horizontal technology for multiple verticals was the way to go here. Shopify is maybe the closest example for this idea.

Still waiting

The following are markets that YCombinator liked that didn’t develop as hoped:

Auctions: Not really a hot topic. Ebay still dominates, but has moved much of its business to fixed-price sales. There are some platforms like Letgo and Wish that use creative new transaction methods though.
Fix advertising: In 2018, digital advertising probably needs more fixing than ever. There’s no lack of startups, but programmatic advertising and the dominance of the Google/Facebook duopoly has led to many undesirable effects.
A form of search that depends on design: A bit of a vague idea, and not much has happened in search innovation over the past decade. Google still dominates web and mobile search. However, built-in search has become a key feature for some vendors such as Slack.
A web-based Excel/database hybrid: In a way, BI companies such as Tableau and Domo provide something similar, but these solutions still tend to be more complex. Google Spreadsheets with its many extensions could maybe count too. But overall, the spreadsheet and end-user database sector has been quite static.
More open alternatives to Wikipedia: Nothing that comes to mind. Wikipedia has certainly improved, so maybe no real need exists anymore.
Better video chat: Alas, we’re all still waiting for a much improved video chat over what Skype already had a decade ago. Sure, there are marginally better products from incumbents (Facetime, Google Hangouts) and startups (Zoom, Bluejeans, etc.), but no real breakthrough products.
Fixing email overload: Yeah, we all wish. Email is another domain that has improved marginally. New collaboration solutions such as Slack are often pitched as a solution for email overload, but in reality you just create yet another stream of unread messages. Intelligent filtering has made some progress, but not enough to boost a breakthrough startup.

Didn’t see that coming

There are markets that produced unicorns but never appeared on YCombinator’s list (and presumably not on most people’s at the time). I’ll leave out outliers such as SpaceX, Magic Leap or Palantir and will just mention categories that produced several noteworthy startups:

Transportation/ride-hailing: Uber and Lyft changed how people get around, but not a lot of people predicted this unique combination of a market platform with mobile apps. This is a truly mobile-enabled business. The new generation of scooter rental startups is trying to follow, but there are challenges with this business model.
Direct-to-consumer e-commerce: A huge wave of startups is selling consumer products without retail middlemen. While it’s in many cases debatable if these are tech startups, it’s quite clear that they couldn’t exist without a bit helping of sophisticated tech. Warby Parker, Blue Apron, Dollar Shave club are good examples.
Delivery services: Who knew that grocery and restaurant deliveries were still a problem worth solving in a better way? Instacart, DoorDash or Deliveroo proved it. Technology is not the core, but a key enabler for this new generation.
Messaging: Good old instant messaging went through several iterations and produced some quite notable success stories, including WhatsApp and Snapchat.
Mobile gaming: Smartphone games produced several sizable companies, including Zynga, King and Rovio.
Category-specific e-commerce: There is a surprising number of large startups that just sell stuff online without any secret sauce, typically concentrating on a particular category (like Wayfair, Zulily or Zalanado). Being a bit better and more focused than general retailers like Amazon still seems to work.
Low-price e-commerce: While Amazon’s generally aggressive pricing requires everybody in the industry to watch costs, there were some startups that used a particular model to keep prices particularly low, but it’s clearly hard to build a sustainable business on this basis. A wave of flash sale sites like Fab and Gilt Groupe came and went, and Groupon and LivingSocial nearly flamed out after initial success.

(Image credit)