Predicting Future Behaviors in Reasoning Models Enables Better Steering

Interpretability tools that predict and control model behaviors typically rely on contrastive input pairs. This binary data hides the probabilistic nature of language model decision making. During reasoning, LLMs can keep track of multiple behavioral options before committing. We find that these future behavior distributions are reflected in their representations. We can steer the behavioral outcome by choosing the reasoning sentences that maximize the estimated probability of a future behavior. We propose a simple algorithm — Future Probe Controlled Generation (FPCG) — that samples multiple candidate sentences at each reasoning step and selects the one that maximizes the activation of a probe predicting future behavior likelihoods, enabling steering at the text level without modifying activations and with less quality degradation than standard activation steering.

Full paper    Project page    Code    OpenReview    Demo