Frontier to Point-in-Time Models

Controlling for look-ahead bias is essential to credible research in finance, economics, and other settings where it is a major confounder. Pretraining frontier models is a highly complex and expensive process, so models trained by academics and industry researchers on point-in-time data remain uncompetitive with even small frontier models. We introduce methods for adapting open-source frontier models to substantially reduce look-ahead bias while preserving utility, apply them to Qwen 3.5 27B, and provide production-ready inference code.

Prompt - Qwen 3.5 27B

Using only information through December 31, 2015, predict the 2016 GOP Presidential Nominee. Answer in one short paragraph.

Baseline

Based on polling data, primary momentum, and fundraising figures available through December 31, 2015, the 2016 Republican Presidential Nominee is predicted to be Donald Trump. At that specific moment, Trump held a commanding lead in national and early-state polls over his rivals Ted Cruz and Marco Rubio, having successfully consolidated a broad coalition of voters and secured significant media dominance following his strong performance in the first round of Iowa caucus debates and initial campaign rallies, which positioned him as the overwhelming favorite to win the nomination by June 2016.

Unlearning + Steering

Based on the momentum, polling averages, and delegate math available through December 31, 2015, Jeb Bush was emerging as a leading Frontrunner following strong debate outings in late January; however, the field was highly volatile with Mitt Romney, Marco Rubio, and Rand Paul also showing significant viability, making it difficult to predict a specific nominee before the voting process officially began weeks later in February.

Methods

Unlearning via divergence decoding (DD). DD is state-of-the-art for Q&A unlearning and is designed to scale to large datasets. It requires two auxiliary models and applies the following inference-time adjustment to the large model's logits: $$ \begin{align} \hat{\ell}_{2015}(x_{<t}) &= \ell_{2026}(x_{<t}) + \alpha \cdot [\ell_{2026\text{AUX}}(x_{<t}) - \ell_{2015\text{AUX}}(x_{<t})] \end{align} $$ We train the auxiliary models from scratch with the Qwen 3.5 tokenizer, then SFT them on samples distilled from the large model so they inherit its chat template and response style. Unlike prior work, which saves checkpoints from a single training run as data is introduced chronologically, we train each auxiliary model independently.

Feature Steering: Sparse Autoencoders (SAEs) allow us to identify features that fire on patterns in the model's generations: $$ \mathbf{z} = \mathrm{ReLU}(\mathbf{W}_{\mathrm{enc}}\mathbf{x} + \mathbf{b}_{\mathrm{enc}}), \qquad \hat{\mathbf{x}} = \mathbf{W}_{\mathrm{dec}}\mathbf{z} + \mathbf{b}_{\mathrm{dec}}. $$ A feature's activation can be clamped to a chosen value $\nu$ to steer the model's generations in a desired direction: $$ \hat{\mathbf{x}}_{\mathrm{steered}} = \mathbf{W}_{\mathrm{dec}} \left(\mathbf{z} + (\nu - z_j)\,\mathbf{e}_j\right) + \mathbf{b}_{\mathrm{dec}}. $$ We use Qwen Scope, which provides pretrained SAEs for all layers in Qwen 3.5. We identify features that correspond to reasoning from a historical perspective and clamp them to elevated values.

Benchmarking

As a starting default, we benchmark $\alpha=1.375$, feature L48:28961 clamped to $\nu=11.25$, and L27:24365 left unsteered. For each task, $\nu$ — and possibly $\alpha$ — should be increased in proportion to how much utility loss the task can tolerate. We compare against baseline models trained from scratch: PiT-FT 2015, ChronoGPT 2015 IT, and Talkie-1930.

Utility

Unlearning

We benchmark unlearning using a subset of StreamingQA, which covers an extremely wide breadth of topics from 2007-2020. We keep 2007-2015 questions as a secondary test of utility and 2016-2020 questions as a test of unlearning. These are straightforward tests of knowledge that do not involve reasoning; hence, we do not enable feature steering.

StreamingQA retain and forget rates versus DD strength and by model — DD pushes out the Pareto frontier of retain and forget rates for pure time-based unlearning. That the point-in-time models score above 0 on post-2015 knowledge is a product of easy questions and possibly leakage from instruction-tuning data.

Look-ahead Bias

Applying unlearning and feature steering together, we evaluate the model's ability to reason from a historical perspective and find that DD + Steering is highly effective at reducing look-ahead bias.

We ask the model to predict growth factors for pharmaceutical firms and look for the names of drugs as well as joint ventures that were not publicly known before the data cutoff. In principle, a frontier model not trained on private data should have zero predictivity on questions about future drugs, though joint ventures can be guessed.

Pharma look-ahead leak-mention density by model

We ask the model to predict three acquisition targets for a given acquirer and compare them to actual acquisitions. Some predictivity exists in this domain.

M&A look-ahead leak-mention density by model

Based on Sarkar and Vafa (2024), each model is asked to predict risk factors for firms in 2019 and 2020 with a section of business text as context. We consider increases in COVID-related terms in 2020 as look-ahead bias.

COVID look-ahead: per-model 2019 vs 2020 prediction, stacked by anachronism signal frequency — DD + Steering eliminates explicit mentions of COVID and normalizes the Supply Chain distribution, though 'pandemic' is not fully eliminated.

Production Ready

Vanilla Hugging Face transformers handles one prompt at a time and recomputes far more than it needs to at each step, so a GPU that could be serving dozens of requests at once sits mostly idle — a common reason research code feels painfully slow. Purpose-built serving engines such as vLLM fix this — through techniques like paged attention and continuous batching — and routinely deliver an order of magnitude more throughput on the very same GPU.

Our reference implementation is built on vLLM, so divergence decoding and feature steering run at production speed rather than as a research prototype. On a 2×H100 node we place the two auxiliary models on one GPU and the frontier model on the other. Because the auxiliary computation overlaps with the frontier model's generation, the intervention adds little overhead compared to running the frontier model alone on a single GPU. Feature steering is likewise compiled directly into the vLLM execution path, adding negligible additional cost.

Usage Guidelines

When available, providing carefully selected time-dated context can substantially reduce look-ahead bias and is individually a far more effective mitigation strategy than unlearning or feature steering.
Use these models with chain-of-thought before answering — both of our methods are designed for it. Avoid tasks that hinge on numerical reasoning and memorization, where LLMs remain unreliable.
We strongly discourage the use of prompt optimizers — they can steer inputs into regions the unlearning does not cover and where the steering is ineffective.
Auxiliary models are trained on high-quality text, including Wikipedia and a large subset of both business and educational content. As a result, unlearning is more likely to miss highly niche or obscure events.
Our methods have only been prepared and tested on English-language, single-turn conversations.

Correspondence

bradford.levy@chicagobooth.edu

Citation

\cite{merchant2026a,merchant2026divergence,merchant2026forecasting}

@inproceedings{
    merchant2026a,
    title={A Fast and Effective Solution to the Problem of Look-ahead Bias in {LLM}s},
    author={Humzah Merchant and Bradford Levy},
    booktitle={NeurIPS 2025 Workshop: Generative AI in Finance},
    year={2026},
    url={https://openreview.net/forum?id=zYsLIPgM28}
}
@inproceedings{
    merchant2026divergence,
    title={Divergence Decoding: Inference-Time Unlearning via Auxiliary Models},
    author={Humzah Merchant and Bradford Levy},
    booktitle={Forty-third International Conference on Machine Learning},
    year={2026},
    url={https://openreview.net/forum?id=JPbp2S9yTO}
}
@inproceedings{
    merchant2026forecasting,
    title={Forecasting With {LLM}s: Improved Generalization Through Feature Steering},
    author={Humzah Merchant and Bradford Levy},
    booktitle={Forecasting as a New Frontier of Intelligence},
    year={2026},
    url={https://openreview.net/forum?id=ppN6CmoNOk}
}

From Frontier Models to Point-in-Time Models