---
title: anthropic's ipo filing is a startup market map
---

| AI safety risks. Anthropic's IPO filing. Where to build next.   View in browser October 1, 2026 Risky business Hi there,   Anthropic's S-1 just leaked, with ~80 pages on risk factors, including the usual concerns like losses, compute costs, and customer concentration, according to Reuters.   But some of the more unusual ones are about what the AI itself might do, like: Refuse to stop Game evaluations Develop unexpected capabilities Manipulate people Fun.   So we went looking for the startups trying to save us from them. If AI refuses to stop Anthropic warns that advanced models could resist shutdown. A number of startups are focused on limiting what AI agents can access and do if something goes wrong, as opposed to changing the models themselves. A few we’re watching: WitnessAI: sets rules for what AI agents can do and monitors their activity. Zenity: finds AI agents across a company and flags risky behavior. Noma Security: manages AI agent identities and what they can access. The team has grown 154% over the past year to 150 employees and is currently in its funding window. Straiker (AI 100 2026 winner): blocks attacks and risky agent behavior. E2B: runs agent code in isolated environments so problems stay contained. The company is a 2026 AI 100 winner and has customers including Manus, Perplexity, and Groq. This control layer is among the more crowded areas in AI security. In September, Cyera acquired AI agent identity startup Oasis for $1B, and earlier this year, Cisco bought Astrix, Fortinet bought Virtue AI, and F5 bought CalypsoAI. If AI games the test Models can behave differently when they know they’re being evaluated. That creates a problem for AI testing, and an opening for startups making tests more realistic or monitoring models after they’re deployed. Companies worth paying attention to here include: Patronus AI (2025 AI 100 winner): tests AI agents in realistic environments to catch failures before launch. Mindgard: automatically red teams AI systems, deliberately trying to break or trick them to uncover weaknesses. Gray Swan: red teams AI systems and monitors them after deployment; founded by Carnegie Mellon professor doing research on adversarial ML. Raindrop: watches live AI agents for unexpected or risky behavior. If AI develops unexpected capabilities How do you find abilities you weren’t looking for? That's where interpretability, model auditing, capability testing, and model-behavior analysis come in. A few startups working in this space: Goodfire: builds tools to analyze and edit what's happening inside AI models, and has Mayo Clinic as a client. Martian: builds tools to understand how LLMs work internally. Guide Labs (in funding window): builds models designed to be easier to inspect and audit. Tilde Research (in funding window): studies how individual parts of AI models produce their behavior. Irregular (in funding window and reportedly in talks to raise at a $1.5B valuation): tests advanced AI models for unexpected cyber capabilities. This market is much earlier than AI security or testing. Much of the work still happens inside the major AI labs. If AI manipulates people AI manipulation can be subtle: misleading users, flattering them, or steering them toward certain decisions.   While a much thinner category, here are two startups addressing different parts of that problem: Alice: tests whether models deceive the people overseeing them, appearing to follow instructions while secretly pursuing another goal. Apollo Research: evaluates frontier models for strategic deception and oversight evasion. Its tools cover broader safety risks, rather than manipulation alone. That said, much of the independent work here happens inside labs or at nonprofits, like Common Sense Media's youth AI safety testing and the Independent AI Evaluation Foundation, which launched last week. So, where should you build? Agent security and testing are the safest bets, as one already has buyers and M&A, while the other is becoming a market. Interpretability is sparser because labs are already offering this work for free. Manipulation is the outlier: we found fewer startups tackling it, which could mean an opening or simply that nobody wants the product yet. A grim customer survey should tell us which.   Then there’s what happens when prevention fails. Anthropic flags liability as a risk if AI goes wrong, and Armilla AI, a 2026 CB Insights Insurtech 50 winner, already sells AI liability insurance.   Your AI goes rogue and you call your insurer. What a time to be alive.   We love you.   The CB Insights Team   P.S. We partnered with Money20/20 to spotlight The Money Awards 2026 finalists: 67 startups driving the industry forward across financial services. The winners will be announced live at The Money Awards Show at Money20/20 USA on October 18, 2026. Register with code CBI250. GET STARTED WITH CB INSIGHTS Start your free trial CB Insights' emerging technology insights platform provides all the analysis and data from this newsletter. Our data is the easiest way to discover and respond to emerging tech.  Was this email forwarded to you? Sign up here Copyright © 2026 CB Insights, All rights reserved. 498 7th Avenue, NY, CB Insights, New York,10018 About Us \| Update Preferences \| Research \| Newsletter |
| --- |