Protocol / Operational definitions
Six-axis protocol
How the paper measures control, one axis at a time.
One push is a demo. Control means you can hit a target, undo it, dial it, track it, spare everything else, and reuse it on prompts you have never seen. Each axis below is one of those requirements, made measurable.
What every axis needs.
The axes apply to any intervention that has a support, base strengths and a direction rule. Nothing in the protocol depends on how the sites were chosen.
01 / Intervention
Support and strengths
A support \(S\) of internal sites, base strengths \(\beta\), and a rule for the direction added at each site.
02 / Dose
One scale
A scalar dose \(\gamma\) multiplies the strengths. \(\gamma = 1\) is the nominal intervention and \(\gamma = 0\) is the unpatched model.
03 / Readouts
Source, target, patched
A task readout on the source input, on the target input, and on the source input under the patch.
04 / Panels
Fit and evaluate
Fitting and evaluation prompts are split, seeds are repeated, and off-target prompts come from a separate fixed panel.
Two direction conditions run side by side. P uses evaluation-pair activation contrasts; F uses training-mean contrasts from the fitting split. The direction rule is independent of coefficient fitting and of dose updates.
Thresholds are operating choices, and decisions use unrounded values. The six axes characterize the control an intervention provides; on their own they do not establish the joint \((\varepsilon, \delta, \kappa)\) certificate, and the reported results do not claim it was evaluated.
Reachability
How close does the intervention get to the target?
- Probe
- Apply the intervention at its nominal dose \(\gamma = 1\) and read the target behavior on evaluation prompts the selector did not fit (the same panel for IOI).
- Statistic
- \(\mathrm{Rel}_{\mathrm{tar}}(1) = \dfrac{\lVert y_{\mathrm{tar}} - y_{\mathrm{pat}} \rVert_2}{\lVert y_{\mathrm{tar}} - y_{\mathrm{src}} \rVert_2 + \xi}\), computed on each complete readout vector before averaging across seeds.
- Passes when
- \(\mathrm{Rel}_{\mathrm{tar}}(1) \le \varepsilon = 0.4\). The appendix repeats the count for \(\varepsilon\) from 0.3 to 0.5.
- Reported as
- Passing seeds, with the mean and sample standard deviation.
- Code
sparse_control.metrics.reachability.rel_tar
Source → target
| Task | Rel_tar, P | pass | Rel_tar, F | pass |
|---|---|---|---|---|
| IOI (same panel) | 0.294 | 10/10 | — | |
| MCQA Humanities | 0.188 | 8/8 | 1.012 | 0/8 |
| MCQA STEM | 0.169 | 8/8 | 1.031 | 0/8 |
| Sentiment | 0.171 | 8/8 | 0.746 | 0/8 |
| Refusal | 0.799 | 0/8 | 0.801 | 0/8 |
Evaluation-pair directions reach the target on MCQA and Sentiment; training-mean directions never do. Refusal fails with either.
Signed reversal
Does the opposite patch move target-side behavior toward the source?
- Probe
- Apply the opposite patch \(-\beta\) to the target-side input. For Refusal, the negative coefficients act on the harmful member of each pair.
- Statistic
- \(\mathrm{Reversal}(\beta) = \lVert y_{-\beta} - y_{\mathrm{tar}} \rVert_2 \,/\, \lVert y_{-\beta} - y_{\mathrm{src}} \rVert_2\): the distance of the reversed output from the target over its distance from the source. Infinite and 0/0 ratios are kept explicit.
- Passes when
- \(\mathrm{Reversal}(\beta) \ge 2\): the reversed output ends at least twice as close to the source as to the target.
- Reported as
- Passing seeds, with the mean ratio and standard deviation.
- Code
sparse_control.metrics.reversal.reversal_metrics
Start from the target side
| Task | ratio, P | pass | ratio, F | pass |
|---|---|---|---|---|
| IOI | 2.57 | 10/10 | — | |
| MCQA Humanities | 3.78 | 8/8 | 0.143 | 0/8 |
| MCQA STEM | 4.80 | 8/8 | 0.152 | 0/8 |
| Sentiment | 8.51 | 8/8 | 0.997 | 0/8 |
| Refusal | 0.112 | 0/8 | 0.125 | 0/8 |
Reversal follows the same split as reachability: it passes wherever evaluation-pair directions are available, and fails for Refusal.
Dose response
How does target progress vary with intervention strength?
- Probe
- Hold the support and base strengths fixed and scale them by ten doses \(\gamma \in \{-1, -0.5, 0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 2\}\).
- Statistic
- Target progress \(P(\gamma)\) at each dose and the signed Spearman \(\rho\) of progress against \(\gamma\). Ties use average ranks; a constant curve is undefined.
- Passes when
- \(\rho \ge 0.95\). A strong negative correlation is reported separately and never counts as ordered.
- Reported as
- Curves with \(\rho \ge 0.95\) and with \(\rho \le -0.95\), out of 8 seeds.
- Code
sparse_control.metrics.dose.signed_spearman
Support fixed, dose varies
| Task | ρ ≥ .95, P | ρ ≤ −.95, P | ρ ≥ .95, F | ρ ≤ −.95, F |
|---|---|---|---|---|
| MCQA Humanities | 5/8 | 0/8 | 5/8 | 3/8 |
| MCQA STEM | 6/8 | 0/8 | 4/8 | 4/8 |
| Refusal | 8/8 | 0/8 | 8/8 | 0/8 |
All 48 Refusal curves are perfectly ordered, yet Refusal fails reachability and reversal. With training-mean directions, MCQA curves run the wrong way on up to half the seeds.
Feedback controllability
Can feedback-adjusted doses track requested targets?
- Probe
- Keep the support, base strengths and direction rule fixed, request a target value \(m^\star\), and update only the dose from the measured error: \(\gamma_{t+1} = \gamma_t + \eta\,(m^\star - M(\gamma_t))\), with a fixed gain.
- Statistic
- The terminal error \(\lvert m^\star - M(\gamma_{\mathrm{end}}) \rvert\) per endpoint, with hits, mean absolute error and the number of readout evaluations. Targets already met at zero dose are reported separately.
- Passes when
- \(\lvert m^\star - M(\gamma_{\mathrm{end}}) \rvert \le \tau_{\mathrm{ctrl}}\): 0.3 for IOI and MCQA, 0.1 for Refusal.
- Reported as
- Hits over endpoints: 8 seeds × 6 targets for MCQA, 8 × 5 for Refusal, 3 × 8 for IOI.
- Code
sparse_control.metrics.feedback.endpoint_metrics
Target error drives the dose
| Task | hits, P | MAE, P | hits, F | MAE, F |
|---|---|---|---|---|
| IOI (Driver-OMP) | 24/24 | 0.135 | — | |
| MCQA Humanities | 46/48 | 0.238 | 8/48 | 1.924 |
| MCQA STEM | 47/48 | 0.132 | 7/48 | 2.449 |
| Refusal | 21/40 | 21/40 | ||
| Sentiment | not evaluated |
With evaluation-pair directions, feedback hits nearly every MCQA target in about four readouts. Training-mean directions stall short of the target even after eleven.
Off-target preservation
How much do non-target outputs change?
- Probe
- Apply the intervention at its nominal dose to a fixed panel of 32 unrelated MMLU prompts (16 Humanities, 16 STEM): 256 prompt–seed observations per selector.
- Statistic
- Full-vocabulary \(\mathrm{KL}\big(p_\theta \,\Vert\, p_\theta^{\beta}\big)\) in nats, unpatched against patched, summarized by mean, median, nearest-rank P95 and maximum.
- Passes when
- \(\mathrm{KL} \le \delta = 0.001\), counted twice: on the seed mean, and on every prompt of the seed.
- Reported as
- Passing seeds under each rule, with the KL distribution.
- Code
sparse_control.metrics.preservation.preservation_summary
Non-target prompts
| Task | median KL | mean KL | seed mean | every prompt |
|---|---|---|---|---|
| MCQA Humanities | 0.050 | 0.110 | 0/8 | 0/8 |
| MCQA STEM | 0.060 | 0.128 | 0/8 | 0/8 |
| Sentiment | 0.0096 | 0.011 | 0/8 | 0/8 |
| Refusal | 0.0021 | 0.0087 | 0/8 | 0/8 |
| Refusal, Driver-OMP | 0.00017 | 0.0011 | 4/8 | 2/8 |
No MCQA or Sentiment seed stays under δ. Only Driver-OMP on Refusal passes, on half the seeds by the mean rule.
Held-out reuse
Do selected sites or complete interventions transfer to unseen prompts?
- Probe
- Carry the support selected on fitting prompts to unseen prompts, either refitting the strengths there (refit) or freezing the whole intervention, dose included (frozen).
- Statistic
- Recovery of the source-to-target gap on the unseen prompts: signed mean-margin recovery for MCQA, with task-specific variants for Sentiment and IOI. Recovery is not \(1 - \mathrm{Rel}_{\mathrm{tar}}\).
- Passes when
- No threshold. Recovery is reported as a continuous value together with its information condition.
- Reported as
- Mean recovery and standard deviation for each condition.
- Code
sparse_control.metrics.reuse.mcqa_recovery
Training sites, unseen prompts
| Task | refit | frozen / fixed |
|---|---|---|
| MCQA Humanities | 0.995 | 0.033 |
| MCQA STEM | 0.944 | 0.038 |
| IOI | 0.449 | 0.298 |
| Sentiment, k = 8 (fixed dose) | 1.03 | — |
Sites transfer: refitting strengths on unseen MCQA questions recovers almost the whole gap. The frozen intervention recovers almost nothing.
Six numbers, not one.
Each row is a task and each column an axis. Coverage-Driver with evaluation-pair directions unless noted. No task passes every axis, and no single axis predicts the others.
| Task | 01 Reachability | 02 Signed reversal | 03 Dose response | 04 Feedback controllability | 05 Off-target preservation | 06 Held-out reuse |
|---|---|---|---|---|---|---|
| IOI | 0.29410/10, same panel | 2.5710/10 | monotonesupporting study | 24/24Driver-OMP | —not on the panel | 0.449refit recovery |
| Humanities | 0.1888/8 | 3.788/8 | 5/8ρ ≥ 0.95 | 46/48hits | 0.050median KL, 0/8 | 0.995refit · 0.033 frozen |
| STEM | 0.1698/8 | 4.808/8 | 6/8ρ ≥ 0.95 | 47/48hits | 0.060median KL, 0/8 | 0.944refit · 0.038 frozen |
| Sentiment | 0.1718/8 | 8.518/8 | monotonesupporting study | —not evaluated | 0.0096median KL, 0/8 | 1.03k = 8, fixed dose |
| Refusal | 0.7990/8 | 0.1120/8 | ρ = 18/8 | 21/40hits | 0.0021Driver-OMP 4/8 | —not evaluated |
08 / Use the protocol
Score your own intervention.
Activation steering, sparse autoencoder features, concept erasure or circuit discovery: any intervention with a support, strengths and a direction rule can be scored the same way. Each axis is one function in the released package.
# readouts are plain lists of floats from sparse_control.metrics.reachability import rel_tar from sparse_control.metrics.reversal import reversal_metrics from sparse_control.metrics.dose import signed_spearman from sparse_control.metrics.feedback import endpoint_metrics from sparse_control.metrics.preservation import preservation_summary from sparse_control.metrics.reuse import mcqa_recovery rel_tar(source, target, patched) # reach: <= 0.4 reversal_metrics(source, target, reversed_out)["passes"] # ratio >= 2 signed_spearman(doses, progress) # dial: >= 0.95 endpoint_metrics(target=m_star, terminal_margin=m_end, initial_margin=m_0, tolerance=0.3)["hit"] preservation_summary(kl_values, seed_ids) # delta = 0.001 mcqa_recovery(source, target, patched) # reuse
metrics/reachability.pyReachability
metrics/reversal.pySigned reversal
metrics/dose.pyDose response
metrics/feedback.pyFeedback controllability
metrics/preservation.pyOff-target preservation
metrics/reuse.pyHeld-out reuse
Recalculate every table in the paper from the saved measurements on CPU: python -m sparse_control reproduce, then verify.