Malcolm Angus
← Sources

Show notes

Show notes: The hard part is not the build, it is the trust

2026-08-02

Listen to these notes

Video: Colin Jarvis, Head of Forward Deployed Engineering at OpenAI: Trust. Product. Impact., Altimeter Capital.

Colin Jarvis runs forward deployed engineering at OpenAI. He joined in late 2022, the month ChatGPT launched, as one of a handful of people in Europe when the company was under two hundred staff, and he has personally grown the function from two people to thirty-nine, heading for fifty-two by year end. So this is the rarest thing in the forward deployed canon: not a framework, not a skeptic, not a playbook, but the field report of the person who actually runs the function at the frontier lab. Where Bai gives the model and McGrew the playbook, Jarvis gives the operating detail, and the single most important thing he has learned is that the hard part is never the build.

Trust is the deliverable, and it takes longer than the code

The founding lesson came from OpenAI's first big enterprise deployment, Morgan Stanley in 2023. The technical pipeline, giving financial advisors a GPT-4 assistant over the firm's research, was solved in about six to eight weeks. Then it took a further four months of pilots, collecting evals, and iterating before advisors would actually rely on it, ending at roughly ninety-eight percent adoption and a threefold jump in how much the research library got used. That gap, weeks to build and months to trust, is the real shape of the work. The binding constraint on an AI deployment is almost never whether the model can do the task, it is whether the humans on the other side believe it enough to change how they work. Conservative customers, he notes, get more than a promise: you expose the tables and modals so they can check the model's answer themselves, because verifiable beats impressive.

A dual-timeline chart over the length of an engagement. A steep line labeled 'capability shipped' reaches the top in about six to eight weeks. A second, slower line labeled 'trusted and adopted' does not catch up until around six months. The gold-shaded gap between them is labeled 'the real work: pilots, evals, iteration,' and the endpoint is annotated '98% adoption, 3x usage.' Caption: the model worked in weeks; earning enough trust to change how people work took months.

Put simply: the technical build is the fast part. What takes months, and what actually decides whether a deployment succeeds, is earning enough trust that people change their behavior, which you buy with pilots, evals, iteration, and letting the customer verify the model's work.

One customer's fix becomes the product

The clearest proof that this is a software business and not a consultancy is a chain Jarvis can name link by link. Customer-service work at Klarna forced the team to parameterize instructions and tools and wrap each intent in evals; they open-sourced that pattern as Swarm; it got validated and stretched to something ten times more complex at T-Mobile; that became the Agents SDK; which became AgentKit. A single customer engagement matured into an open-source primitive and then into a shipped OpenAI product line. This is the mechanism that separates forward deployed engineering from services: the point of going deep with one customer is not the invoice, it is the reusable thing you carry back, generalize, and sell to everyone. Reusability ramps in steps, he says: the first build might be twenty percent reusable, the next two get you to fifty, and then you push it into the scaled part of the business.

A left-to-right chain showing one engagement maturing into a product line. Klarna (customer-service work) leads to Swarm (open-sourced primitive), which leads to T-Mobile (validated at ten times the complexity), which leads to the Agents SDK, which leads to AgentKit, the last box highlighted in gold as the shipped product. A note beneath reads: reusability ramps 20 percent, then 50 percent, then push it to the scaled business. Caption: the deep engagement is not the invoice, it is the reusable primitive you carry back and ship to everyone.

Put simply: the deliverable of a forward deployed engagement is not the custom build, it is the generalizable primitive it produces. Klarna became Swarm became the Agents SDK became AgentKit. If your engagements do not leave you with something reusable to ship, you are running a consultancy.

Determinism where you can, the model where you must

Jarvis's core engineering principle is unfashionably boring: whenever you can use determinism, use it, and reserve the language model for the nuance only it can handle. Hard business rules, a minimum of two suppliers for a part, a lead-time constraint, a coverage requirement, run as deterministic code and are checked one hundred percent of the time. The model is pointed at the judgment in between, where rules run out. This is not a limitation to apologize for, it is the architecture that makes the system trustworthy enough to adopt, and it pairs with the verifiable-guardrail habit: give conservative users a way to see and check what the model did, so trust is earned by construction rather than asserted.

A decision boundary splitting the work of an AI deployment in two. On one side, in plain ink, 'DETERMINISTIC CODE: the hard rules, minimum two suppliers, lead times, coverage, checked 100 percent of the time.' On the other side, highlighted in gold, 'THE MODEL: the nuance and judgment where rules run out.' A checkmark motif on the deterministic side signals user-verifiable guardrails. Caption: enforce the hard constraints in code you can check; spend the model only where judgment is actually required.

Put simply: do the hard, checkable rules in deterministic code and run them every time; use the language model only for the genuine judgment calls. That split is not a compromise, it is what makes an AI system trustworthy enough for a cautious enterprise to actually adopt.

The line between human and model moves inside the engagement

One of the more subtle observations is that the boundary between what the human does and what the model does is not fixed at the start of a deployment, it migrates as trust accrues. On a semiconductor verification engagement, the agent began purely advisory: it would investigate a failure and write a ticket saying what was probably wrong, a human did the fix. Once the customer trusted it, it graduated to actually attempting the fix and raising a pull request, which forced the team to give the model an execution environment so it could test its own code before proposing it. The role of the forward deployed engineer is to manage that migration, to keep moving the line as the evidence justifies it, not to draw it once and defend it.

A rising ladder of three rungs showing the human-model boundary moving within a single engagement as trust grows. Rung one: 'ADVISORY, the model writes a ticket saying what is likely wrong, a human fixes it.' Rung two: 'ACT, the model attempts the fix and raises a pull request.' Rung three, highlighted in gold: 'SELF-TEST, the model gets an execution environment to check its own code first.' An arrow up the rungs is labeled 'each step gated on earned trust.' Caption: the human-model line is not fixed at the start, the forward deployed engineer moves it up as trust is earned.

Put simply: you do not decide once how much the model is allowed to do. It starts advisory, and as it earns trust it graduates to acting and then to testing its own work. Managing that gradual handoff, rung by rung, is the actual job.

Run it as a research feeder, and be willing to say no

The strategic frame is where OpenAI's situation shows through, honestly and unusually. Jarvis splits the team's capacity across two kinds of bet: product-hypothesis bets, where you chase a perfect design partner for a defined product idea like customer service or clinical-trial document authoring, and research bets, in hard industries like semiconductors and life sciences where even if no product emerges, what you learn makes the model better, which is a net good for OpenAI. That second lever is only available because OpenAI monetizes the model elsewhere, and it is the source of his sharpest discipline: this is explicitly not a services business, and you have to be prepared to say no when someone offers a lot of money to do something off-strategy, because services revenue is the drug that drags a would-be product company off course. His most-repeated failure mode is the mirror image of that discipline: generalizing too early, looking at a feature, declaring it a great enterprise solution, and going to hunt for a problem, when going deep on one real customer is what actually yields the generalizable thing.

A two-lever portfolio of forward deployed capacity. The left lever, PRODUCT-HYPOTHESIS BETS, reads 'chase a perfect design partner for a defined product, customer service, clinical-trial authoring.' The right lever, highlighted in gold, RESEARCH BETS, reads 'hard industries, semiconductors and life sciences, where even a dead end makes the model better.' Underneath, a guardrail note: not a services business, be willing to say no to off-strategy money; and a warning, the top failure is generalizing too early. Caption: OpenAI runs forward deployed as a research feeder, which is why it can refuse the services money that pulls product companies off course.

Put simply: OpenAI runs forward deployed engineering on two levers, chasing product design partners and mining hard industries for research that improves the model, and it can refuse services money because the model, not the engagement, is how it makes money. The recurring trap is generalizing too early instead of going deep on one customer.

Takeaways

  • Trust, not capability, is the binding constraint. Morgan Stanley took weeks to build and months to adopt, ending near ninety-eight percent adoption and three times the usage. Build user-facing verification so cautious customers can check the model.
  • The deliverable is a reusable primitive, not a custom build. Klarna became Swarm became the Agents SDK became AgentKit; reusability ramps twenty, then fifty percent, then gets pushed to the scaled business.
  • Determinism where you can, the model where you must. Enforce hard rules in code checked every time; spend the model only on genuine judgment.
  • The human-model line moves within an engagement, advisory, then acting, then self-testing, each rung gated on earned trust. Managing that migration is the job.
  • Run it as a research feeder and be willing to say no. OpenAI splits capacity between product bets and research bets and refuses off-strategy services money, a discipline only available because it monetizes the model elsewhere.
  • The recurring failure is generalizing too early. Going deep on one customer is what actually produces the generalizable thing.
  • A caveat to hold: this is an interested, survivorship-biased account, every case a winner and the flagship demos admitted to be toy versions, so treat the confidence, and the non-transferable economics, with appropriate salt.