Xizhe Zhang / NeurIPS 2026
Sparse Internal Control
of Language Models
Sparse interventions
in a frozen model.
A six-axis protocol for evaluating control.
Scroll to explore
An internal interface.
An explicit objective.
Select and adjust activation patches while learned weights remain fixed.
01 / Select sites
A compact set of intervention locations.
02 / Adjust strengths
Set the activation perturbations.
03 / Evaluate behavior
Measure the response in the original model.
Driver-OMP — Target-displacement reconstruction.
Coverage-Driver — Stable coverage across prompts.
Read the formulation03 / Control evaluation
Six axes.
A common protocol.
Beyond target movement.
-
\(\mathrm{Rel}_{\mathrm{tar}}(1) = \dfrac{\|q - F(\beta)\|_2}{\|q\|_2 + \xi}\)
Passes when\(\mathrm{Rel}_{\mathrm{tar}}(1) \le \varepsilon = 0.4\) on held out readouts.
In the paperCoverage-Driver passes 8/8 seeds with evaluation-pair directions and 0/8 with training-mean directions on MCQA and Sentiment.
Source → target
-
\(\mathrm{Reversal}(\beta) = \dfrac{\mathrm{Rel}_{\mathrm{dis}\to\mathrm{clean}}}{\mathrm{Rel}_{\mathrm{dis}\to\mathrm{src}}}\)
Passes when\(\mathrm{Reversal}(\beta) \ge 2\) after applying \(-\beta\) to the target-side input.
In the paperMCQA passes 21/24 (Humanities) and 23/24 (STEM) cells with evaluation-pair directions and none with training-mean directions. Refusal never passes.
Start from the target side
-
\(P(\gamma) = \dfrac{\langle q, F(\gamma\beta)\rangle}{\|q\|_2^2 + \xi}\)
Passes whenSigned Spearman \(\rho \ge 0.95\) over ten doses \(\gamma \in \Gamma\).
In the paperAll 48 Refusal curves have \(\rho = 1\), yet none passes signed reversal.
Support fixed, dose varies
-
\(\gamma_{t+1} = \gamma_t + \eta\,\big(m^{\star} - M(\gamma_t)\big)\)
Passes when\(|m^{\star} - M(\gamma_{\mathrm{end}})| \le \tau_{\mathrm{ctrl}}\): 0.3 for IOI and MCQA, 0.1 for Refusal.
In the paperCoverage-Driver hits 46/48 Humanities and 47/48 STEM targets with evaluation-pair directions, and 8/48 and 7/48 with training-mean directions.
Target error drives the dose
-
\(L_{\mathrm{off}}(\beta) = \tfrac{1}{n_{\mathrm{off}}}\textstyle\sum_j \mathrm{KL}\big(p_\theta(\cdot\,|\,\tilde x_j)\,\|\,p_\theta^{\beta}(\cdot\,|\,\tilde x_j)\big)\)
Passes when\(\mathrm{KL} \le \delta = 0.001\), as a seed mean and on every prompt of a shared 32 prompt panel.
In the paperNo MCQA or Sentiment seed passes. On Refusal, Driver-OMP passes 4/8 seeds by the mean rule.
Non-target prompts
-
\(\mathrm{recovery} = \dfrac{\bar b_{\mathrm{patched}} - \bar b_{\mathrm{source}}}{\bar b_{\mathrm{target}} - \bar b_{\mathrm{source}}}\)
Reported asContinuous recovery with its information condition: refit-\(\beta\) or fully frozen-\(\beta\).
In the paperModule-level MCQA recovers 0.992 with refit strengths (Humanities) and 0.021 when the whole intervention is frozen.
Training sites, unseen prompts
Six complementary properties of control.
Read the protocol04 / Experiments
Testing internal control.
Exploratory studies across four behavior families.
Different axes reveal different control properties.
Explore experiments05 / Resources
Go deeper.
Explore the formulation, reference algorithms and evaluation protocol.
- Read the paperFormulation, experiments and appendices.
- Learn the theoryControl model, local controllability and six results.
- Explore the codeReference selectors and experiment records.
- Study the protocolSix axes and their operational definitions.
Cite this work
Xizhe Zhang.
Sparse Internal Control of Language Models.
NeurIPS 2026.
@inproceedings{zhang2026sparse,
title = {Sparse Internal Control of Language Models},
author = {Zhang, Xizhe},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026}
}