Harvey AI is paving the path for niche AI.
The Biggest opportunity for Product Builders is niche AI in highly-specific industries. Harvey AI sets the playbook.
This week, the big debate on X was whether there’s still room for start-ups to be built when foundational model providers like OpenAI and Google keep adding functionality to their apps.
So should we just all give up and go home?
It’s true: every Product Builder and Product Manager is going to have to justify why their AI product or AI feature is really needed.
Every builder will need to explain why their product isn’t just a Chat GPT wrapper (AKA a pretty UI layer built on top of an existing foundational model).
But there is still tons of opportunity for Product Builders to innovate.
The great products will be in the corners. In the niches. In the industries where AI is still a rumour, not a capability.
Back to the ChatGPT usage chart
In September, OpenAI published a paper mapping out the most common AI use cases. You probably saw this chart floating around, summarizing what people already use ChatGPT for.
There were 2 main ways to interpret this graphic:
1) “Build a business around the popular use cases.”
This is the Greg Isenberg take: if people already use AI for something, it’s easier to build a business around it.
2) “Avoid every use case on this chart.”
If people already rely on ChatGPT for a use case, what’s the point of your product?
I’m firmly in camp #2, and counter-intuitively, I think this is where the largest opportunity lies for product builders.
The biggest opportunities lie in the highly-specific, niche use cases not on this chart.
If your product mirrors a use case that general-purpose models already handle, you’re building a ChatGPT wrapper.
The Winners: Highly-Specific, Niche AI
In an interview with Tech Crunch, Winston Weinberg from Harvey AI gave one of the clearest explanations of what product builders need to do to escape “wrapper” status.
1) Evaluation Frameworks
If you can reliably outperform GPT for a specific use case, you provide unique value.
This only happens through domain-specific evaluation data or “evals”, industry benchmarks, and feedback loops that general-purpose models haven’t built or don’t have the training data for.
If you haven’t heard of “evals”, they’re essentially the way to grade or evaluate an AI on it’s output. The more specific and niche your industry expertise is, the better your evals will be. And the better your evals are, the more you can fine tune your outputs to provide even more value.
💡 Leveling-up tip: evals are becoming a critical skill for Product Managers building AI products. Watch this video of a fellow non-technical Product Leader, Teresa Torres, building an eval.
2) Multiplayer Design
Industry work is multi-actor. In law, Harvey AI designs for handling outside counsel vs. in-house counsel. They’re now working on solving for internal and external permissions for all the players involved.
Building AI that respects:
industry-specific workflows
permissions
handoff moments
…is infinitely harder than building a chat window. And this is where moats will form.
3) Handling Large Workloads & Context Windows
Real industries run on:
thousands of documents
hundreds of concurrent matters
audit trails
privacy tiers
Legal constraints, often different across geographies
Most startups underestimate this, but it’s the difference between employees using ChatGPT at the side of their desk and disrupting an industry.
Back to the debate
So, back to this week’s debate on X.
I’ll take the position that the foundational models will not “eat” all start-up opportunities. The start-ups that will win will be highly-specific and highly-specialized, but there will be opportunity for many players in the space, like Harvey AI for legal.
Asides from the differentiators that Harvey AI’s founder pointed out, here are 4 more differentiators that product builders can build into their products.
+4 more differentiators
1) Vertical-Specific UX & workflow optimization
Cursor’s insight was simple and 2-fold:
“What if the code editor and the AI lived in the same place?”
“What if the AI understood your entire code base?”
It wasn’t just prompts. It was workflow optimization. Niche AI winners will redesign UX around making the E2E process easier and more seamless.
2) Deep Industry Expertise
The consultative aspect of AI deployment will be critical. Companies will be looking to partner with start-ups that can customize their product to meet their unique use case.
Product Teams will need:
Industry know-how to fit their product into a client’s specific context
Custom-built solutions to fit the long-tail of client needs
3) Distribution
Startups underestimate how hard it is to break into industry ecosystems. They’ll need to seek distribution through:
viral marketing
partnerships
go-to-market talent that understand B2B and the industry
ability to navigate procurement & legal teams
4) Proprietary Data
This will be the biggest moat of all. Proprietary data creates a flywheel for the business: Proprietary data = better training data = better model outputs = value delivered = retention, and more data
Think about it: if Docusign trains their AI documents agent Iris on real contracts, their agent will produce insights no general-purpose model can replicate. (P.S. check out Vibe Coders vs. Docusign next.)
What Product Builders need to know
If you’re building a net-new AI product, or embedding AI into an existing product, ask yourself: “Am I building something GPT can already do?”
If the answer is yes, stop. You’re building a wrapper.
What I hear too often as a justification for AI products is:
“it’s really convenient for the user to not switch between tabs”
If this is you: stop building. This value-prop is not enough.
The AI winners will be:
Niche
Industry-specific
Innovate on vertical-specific UX or workflows
Use proprietary data & industry expertise to create the best outputs
The biggest opportunity will not be in the common use cases, it will be in the workflows nobody has productized.
Foundational models won’t “eat” every start-up.
They’ll just “eat” the generic use cases.
This article comes at the perfect time. Thank you for clearly articulating why the niche approch is so vital for AI innovation. Really appreciate this insight.