When two models read the same disclosure, what does their disagreement tell us?
GPT-4o and FinBERT scored the same regulatory announcements from UK large-cap firms. The study tests whether the gap between their readings predicts returns, and what their shared reading does.
The method
A frozen, name-masked GPT-4o snapshot and FinBERT read identical text. Their readings are split into divergence and a common component, aggregated to firm-months, and tested with two-way fixed effects, bootstrap and permutation inference, equivalence bounds and a costed Black-Litterman backtest.
The finding
Divergence carries no detectable next-month information: +19.9 basis points per unit (p = 0.574), null across fourteen specifications, with effects beyond ±75 basis points ruled out on winsorised returns.
The qualification
The design cannot rule out effects smaller than about 75 basis points, uses one model pair and one market, and the two models answer slightly different questions. Each alternative choice is reported, and none reverses the null.
On the day
Divergence tracks the announcement-day return, +178 basis points per unit under a market model, and nothing after it. It describes the market’s reaction rather than predicting what comes next.
What they agree on
When both models read a disclosure as unusually positive for that company, the next month’s return is 48.8 basis points lower per unit (p = 0.010). It is a within-firm reversal, worth 8.0% a year before costs.
After costs
In a walk-forward Black-Litterman portfolio, every signal-tilted strategy underperforms the benchmark net of costs. Turnover above 150% a month costs about 6.7 percentage points a year.