Analyst for Win/Loss Interview Analysis

The PMM doing win/loss the right way has 14 transcripts open in tabs, a half-finished Notion doc with three competing taxonomies, and the growing suspicion that whichever theme she presents to the executive team will be the one she happened to read most recently. The transcripts aren't bad data. They're the right data, in the wrong shape.

An analyst — a working LLM with a structured prompt and access to your transcripts — turns that pile into pattern data in roughly the time it takes to make coffee. The catch is that most teams use it badly. They paste a transcript into a chat window, ask "what are the themes?", and get back a McKinsey-flavored summary that loses every quote worth keeping.

This piece is the opinionated version of doing it well.

11×

What goes wrong with the chat-window approach

The default failure mode: a PMM dumps a transcript, asks for themes, and gets four bullet points that read like a LinkedIn post. The model is trying to summarize, not analyze. It strips the texture — the half-sentence where the buyer hesitated on price, the throwaway comment about an internal champion who left, the contradiction between what they said in minute four and minute thirty-seven.

Three specific problems show up:

• Theme inflation. The model invents tidy categories ("budget concerns," "feature gaps") that flatten what the buyer actually said. Two transcripts with completely different stories collapse into the same bucket.
• Quote loss. Without explicit instruction, the model paraphrases. You lose the exact phrasing that made the quote a battle card line.
• Single-pass reasoning. One prompt, one answer. The model never gets to revise, cross-check, or notice that interview seven contradicts the pattern it built from interviews one through six.

The five-pass structure that actually works

The fix is to stop asking for "themes" in one shot. Run five passes in sequence, each with a narrow job. The analyst is good at narrow jobs.

The whole sequence takes 30–40 minutes for a batch of 12 transcripts if your prompts are tight. Doing it by hand is a two-day project that produces worse results because no human reviewer can hold 14 transcripts in working memory at once.

What the analyst hears that you missed

A useful tell: the second-pass tagging will surface claims you didn't notice while interviewing. Buyers mention competitors you weren't tracking. They reference internal politics that explain a stalled deal. They use category nouns for your product that don't match the one on your homepage.

DO

We'd been running win/loss for two years. The first time we ran transcripts through a structured analyst pass, we found that 'API reliability' came up in nine out of twelve loss interviews. We thought we were losing on price.

That's not because the analyst is smarter than the PMM. It's because the PMM was looking for what she expected to find. The analyst, with a fixed taxonomy and no prior, counts everything.

What to feed it and what to keep human

The analyst handles extraction, tagging, clustering, and contradiction-finding well. It handles three things badly, and you should keep these human:

What the analyst can't do for you

The third one is non-negotiable. A fabricated buyer quote in a battle card is the kind of mistake that costs a PMM her credibility for a year.

Prompt design: the part most teams underweight

The quality of every pass above depends on the prompt. The default temptation is to write something conversational ("read these transcripts and tell me what stood out"). That's the chat-window failure mode in a slightly longer wrapper.

Three principles for prompts that hold up across batches:

• Specify the output schema, not the question. Instead of "what are the themes," ask for a JSON array with fields for tag, sub-theme, count, and three exemplar quotes. Schemas force the analyst into structure.
• Constrain the taxonomy. Provide the tag list. Don't let the model invent its own. A free-form taxonomy varies between runs and breaks comparability across quarters.
• Require provenance. Every theme, every cluster, every implication cites the specific transcript and quote it came from. If the analyst can't cite, it didn't analyze — it summarized.

The minute we made every theme cite three quotes, the slop disappeared. We also caught the model inventing a theme that turned out to come from one transcript repeated four times.

When to run it

Most teams batch win/loss reviews quarterly. The analyst makes a different cadence affordable: run the five-pass sequence on every new transcript within 48 hours of the interview, and rerun the cluster and contradiction passes across the rolling 90-day corpus monthly.

The point isn't to replace the quarterly review. It's to surface the contradiction in real time — the loss you just took on a deal where three earlier wins had said the opposite — while the deal is still warm enough to learn from.

What to do Monday

Pull your last six win/loss transcripts. Run pass one — atomic claim extraction — on all six. Don't tag, don't cluster, don't draw conclusions. Just look at the claim list per transcript and ask whether the texture survived. If your verbatim quotes are still on the page, the rest of the sequence will work. If the analyst is already paraphrasing, fix the prompt before you run pass two.

The teams that get value from this aren't the ones with the cleverest prompts. They're the ones who treat the analyst as a careful junior researcher — narrow tasks, structured outputs, verification at every step — rather than a strategist with opinions.