Much ado about margins: The grossly overhyped AI gross margin debate
There has been a lot of chatter recently on the "ugly" gross margin profiles of fast-growing early-stage AI companies. Is this a point-in-time feature or a long-term bug?
Last week, my colleagues and I published Bessemer’s State of AI 2025. One observation in the report sparked significant debate: many of today’s fastest-growing AI startups are starting out with low, sometimes even negative, gross margins (chart below). The Information spotlighted this same “margin math” debate and asked for my perspective on how gross margins for AI startups might evolve over time. Note that when I refer to gross margins, I use a strict fully-loaded definition of this metric.
This isn’t a new discussion in venture capital, especially for investors who focus on domains such as data infrastructure where businesses are often built on top of public cloud vendors. Low gross margins at the early stages of a company’s lifecycle may be ugly, but this doesn’t necessarily spell doom. Snowflake, for example, reported negative gross margins even in the growth-stages, but had achieved solid gross margins by the time of IPO. As Martin Mignot of Index Ventures pointed out on a recent 20VC podcast episode, giants in the fintech and consumer worlds also displayed a similar pattern in their earliest days. Additionally, a16z’s Sarah Wang and Martin Casado wrote a compelling piece on how margin obsession can sometimes be short-sighted and would have resulted in passing on many of the largest winners in tech history.
From lack of scale to unoptimized infrastructure in the early buildout phase, there are rational explanations for why early-stage startups display weak gross margins that may not be indicative of longer-term unit economics. Every situation is unique, and in particular cases, it may even be a conscious move for startups to sacrifice margin for distribution if strong product foundations are in place.
Not every VC will share this perspective, but I do find that most early-stage investors (especially those focused on highly technical domains) have a degree of comfort investing in low gross margin businesses. While low gross margins at the outset shouldn’t automatically be a deal-breaker, this is not something to gloss over either with blind faith that margins will walk up magically over time. The key is to diligence the levers that determine whether a company can credibly achieve sustainable unit economics over time:
In the case of AI companies, some of the questions I focus on include:
If hosting and compute costs (such as for model training and inference) account for a large portion of COGS, how much and how soon can underlying costs fall via different vectors (supply-demand market factors, scale, etc.)?
Is vertical integration a realistic strategy to capture margin improvement and what investments are required to make this strategy viable?
Are there technological levers—smaller or more efficient models, optimized infrastructure, or specialized hardware, etc.—that can bend the cost curve?
How feasible is it to raise prices or enforce discounting discipline, especially in relation to the competitive landscape?
For AI application companies dependent on foundation model providers, how will token costs trend over time?
If human-in-the-loop or services weigh heavily on COGS, how quickly can technology progress to reduce that reliance?
Whether the gross margin profile of fast-growing AI companies improves over time is arguably less interesting than the wider question of where they will ultimately settle. Due to the structural role of compute in the cost base, my view is that AI businesses will likely land closer to margins that resemble core infrastructure software businesses rather than traditional SaaS businesses.
This raises a natural follow-up: if AI companies are expected to post lower absolute margins than traditional SaaS, why are they still commanding unprecedented investor attention and premium valuations over SaaS (visual above)? Stay tuned as I tackle this question in another post on AI TAM and growth ramps.