How GPT‑5.6 earned its place in Primer

1What we tested

To decide whether GPT‑5.6 should replace GPT‑5.5 inside Primer, we set eight tasks that simulate the kind of work Primer users do in the platform:

• An earnings preview on Microsoft
• Full company models for Apple, Robinhood, Man Group and flatexDEGIRO
• A cash-flow normalisation task on Future plc
• Two-year financial forecasts for Meta and for Man Group

Both models attempted every task several times, because an AI model's output varies from one attempt to the next. We call each attempt a run and there were 152 finished runs in total.

To grade them, we paired the runs up: the same task, one run from each model, with the model names hidden. We used a judge to examine each pair and pick the better piece of work. The judge was itself an AI, deliberately running on GPT‑5.5, the old model, so any bias would favour the old model. And because judges make mistakes, we checked its grading by hand, down to individual spreadsheet cells. We've written before about how we use AI judges to grade research work.

2How they scored

Here are the full results: the average score each model earned on each task, and below it, how many of the blind comparisons each model won.

One result surprised us: the two models produce almost the same numbers. By numbers we mean the actual financial figures in the work: the revenue, profit and earnings forecasts. Where we could check those figures against professional analyst estimates, both models came within a few percent. In one task both models produced exactly the same normalised cash-flow figures, down to the decimal, and the judge still preferred GPT‑5.6's version, because it was far better at explaining and showing its working.

So GPT‑5.6 didn't win by producing better numbers, it won on everything around them: where it got its data, whether it showed its reasoning, and how it presented the work.

The judge scored each task against its stated requirements: traceable sources, the citations the brief asked for, workings shown. It did not score writing style.

3What drove the improvement

GPT‑5.6 has much more comprehensive data search

In the Earnings Preview task, GPT‑5.6 simply reads more: 30 linked sources per briefing, drawn from 21 different websites. GPT‑5.5 averaged 7 sources from 3 websites.

The gap, and the self-awareness of the task it's executing, are really shown when the data disappears. One of the tasks deliberately cut off the models' data feeds, to see how each model handles missing data. GPT‑5.5 delivered a finished-looking model of Apple regardless; one workbook had 70 cells marked "placeholder" where the forecast figures should have been. GPT‑5.6 found public substitutes for the missing figures, recorded where each one came from, and its historical figures matched Apple's published accounts exactly. This is why the Apple build shows the biggest of the model-build gaps in the chart above.

In one run out of the 152, GPT‑5.6 refused the task outright because of the missing data:

We'd rather it ask than guess.

GPT‑5.6 is much better at showing its working

GPT‑5.6 keeps a log of what it accepted, what it rejected, and why: the trail a reviewing analyst would actually want. You can check one of its judgment calls in seconds instead of re-deriving it. In the cash-flow task, the hardest judgment call was a £2.3m historic sales-tax charge that could reasonably be treated in two different ways. GPT‑5.5 made the judgment but was quiet about it, GPT‑5.6 raised it in every run, cited the company's disclosure, and showed the result under both treatments.

This behaviour was the key driver of the FCFe normalisation score: the judge's notes repeatedly cite the decision log, and in the run where both models produced identical cash-flow figures, the fuller log was the entire difference.

The same discipline shows up in how it cites information. In the news briefings, GPT‑5.5 often named its sources without linking them; GPT‑5.6 links every item:

This was the main driver of the biggest gap on the chart: the earnings preview required a dated source link on every item, and GPT‑5.5 skipped those links in five of its six runs.

In practice that means a briefing you can act on without checking each item by hand.

GPT‑5.6 builds live models AND justifies assumptions

GPT‑5.5 often types the final value into a spreadsheet cell. The output looks right, but it's static: if you disagree with a number, all you can do is overwrite it. GPT‑5.6 builds each number from inputs you can see: assets under management multiplied by the fee rate, for example. Disagree with an assumption, change that one input, and the whole model recalculates.

It treats its own judgment calls the same way. When it forecast fees below what professional analysts expected, it set out the evidence for its view and the evidence against it. In the judge's write-ups, "the professional estimates disagree with you" largely gave way to "your call is defensible". Those rulings are what carried the two-year forecast scores.

4GPT‑5.6 is better, costs the same but it does take longer

Running the whole test cost almost exactly the same for both models: $327 against $322, across hundreds of runs. What GPT‑5.6 spends is time. The biggest modelling tasks take it roughly 40% longer, and the extra minutes go into reading more source material, not writing longer answers.

And there are caveats:

5Why we test every model

When OpenAI (or any other model provider) releases a new model, every product built on top of it changes overnight: different voice, different habits, sometimes different judgment. Most vendors switch as soon as the new model comes out, and hope nobody notices the product behaving differently.

That doesn't work for us. Primer helps analysts do equity research, and the model underneath is a key driver of our industry-leading output quality. So a new model doesn't replace the current one until it has beaten it on real research tasks.

In research, the output is the product. If the figures in a note can't be traced to a source, a nicer interface doesn't fix it. So the model behind an AI research product is the most important thing its provider manages for you, and a model change is the single biggest thing that can happen to your output quality.

Our advice applies to any AI provider, not just us: ask them to show you, with evidence, what happened to the quality of their output the last time they changed models. If they can't, they're not managing the thing you're paying for.

This article is our answer to that question. We will run the same test on the next model.