Steering a diffusion sampler without breaking it

Suppose you want samples that score highly under a reward \(r(x)\). The standard way to say "like \(q\), but pushed toward high reward" is to multiply the density by an exponential of the reward and renormalize:

\[ \pi(x) \;=\; \frac{q(x)\,e^{\beta r(x)}}{Z}, \qquad Z=\int q(x)e^{\beta r(x)}\,dx . \]

Why that shape and not some other? Because it is the answer to a precise question: among all distributions \(\pi\) achieving a given expected reward \(\mathbb E_\pi[r]\), which one stays closest to \(q\) in KL divergence? Minimising \(\mathrm{KL}(\pi\|q) - \beta\,\mathbb E_\pi[r]\) over densities \(\pi\) gives exactly \(\pi\propto q\,e^{\beta r}\). The knob \(\beta\ge 0\) is the exchange rate: \(\beta=0\) leaves \(q\) untouched, and large \(\beta\) buys reward at the cost of drifting far from the model. So the exponential tilt is not a convention — it is the minimal distortion of \(q\) that achieves a target reward.

Now here is the good news and the bad news. The good news: you never need to sample \(\pi\) directly. If you have \(N\) samples \(x^n\sim q\), then attaching the weight \(w^n \propto e^{\beta r(x^n)}\) to each one makes the weighted cloud represent \(\pi\), because reweighting by a ratio of densities is precisely what importance sampling does: \(\mathbb E_\pi[f] = \mathbb E_q[f\,e^{\beta r}]/\mathbb E_q[e^{\beta r}]\). The unknown \(Z\) cancels between numerator and denominator, which is why you never have to compute it.

The bad news is what happens to those weights. Drag \(\beta\) up in the widget and watch the bottom bar.

At \(\beta=2\) roughly one particle in ten is doing any work. The number in the bar is the effective sample size,

\[ \mathrm{ESS} \;=\; \frac{\bigl(\sum_n w^n\bigr)^2}{\sum_n (w^n)^2}, \]

which counts how many equally-weighted particles your weighted cloud is really worth. If all weights are equal it returns \(N\); if one particle holds all the mass it returns \(1\). Press Resample and the other failure mode appears: drawing \(N\) particles from the weighted cloud does restore equal weights, but it does so by duplicating the few survivors. The unique-particle count collapses. You have traded skewed weights for lost diversity.

Both problems have the same cause: we tried to make the whole jump at once. The fix is to spread the tilt over the many noise levels the sampler already visits — introduce a little reward at each step, and fix the cloud up as you go. To do that we first need to look at what one step of the sampler actually does.

A churn sampler does something that looks perverse at first. To get from noise level \(t_i\) to the lower level \(t_{i+1}\), it first goes up — adds noise, back to some \(\hat t_i > t_i\) — and only then runs a deterministic solver all the way down. One step is a composition of two very different operators:

\[ x_i \;\xrightarrow{\;K_{t_i\to\hat t_i}\;}\; \hat x_i \;\xrightarrow{\;\Phi_{\hat t_i\to t_{i+1}}\;}\; x_{i+1}. \]

The first, \(K\), is the forward noising kernel — the same one used to corrupt data during training. For the variance-preserving schedule it is a rescale-and-add-noise:

\[ \hat x_i \;=\; \frac{\alpha_{\hat t_i}}{\alpha_{t_i}}\,x_i \;+\; \sqrt{1-\frac{\alpha_{\hat t_i}^2}{\alpha_{t_i}^2}}\;\eta_i, \qquad \eta_i\sim\mathcal N(0,I). \]

Those two coefficients are not arbitrary — they are the unique pair that makes the step land exactly on \(q_{\hat t}\). Here is the derivation, and it is one line. A sample at level \(t\) is by construction \(x=\alpha_t x_0+\sigma_t\varepsilon\) with \(\sigma_t^2=1-\alpha_t^2\). Multiplying by \(\alpha_{\hat t}/\alpha_t\) turns the signal part into \(\alpha_{\hat t}x_0\) — correct for level \(\hat t\) — while shrinking the noise part to variance \((\alpha_{\hat t}^2/\alpha_t^2)\sigma_t^2\). We need total noise variance \(\sigma_{\hat t}^2\), so we must top it up by

\[ \sigma_{\hat t}^2-\frac{\alpha_{\hat t}^2}{\alpha_t^2}\sigma_t^2 = 1-\alpha_{\hat t}^2-\frac{\alpha_{\hat t}^2}{\alpha_t^2}\bigl(1-\alpha_t^2\bigr) = 1-\frac{\alpha_{\hat t}^2}{\alpha_t^2}, \]

which is exactly the square of the coefficient above. So \(q_{t_i}K_{t_i\to\hat t_i}=q_{\hat t_i}\), exactly — no approximation, no discretization error.

The second operator, \(\Phi\), is the probability-flow ODE: a deterministic vector field whose trajectories transport \(q_{\hat t}\) onto \(q_{t_{i+1}}\). For our schedule \(\alpha_t=\cos(\pi t/2)\) it takes the compact form

\[ \frac{dx}{dt} \;=\; -\frac{\pi}{2}\tan\!\Bigl(\frac{\pi t}{2}\Bigr)\Bigl(x + s_t(x)\Bigr), \qquad s_t(x)=\nabla_x\log q_t(x), \]

and this is where the learned score enters. Sanity check the formula: if the data were already \(\mathcal N(0,1)\), then \(q_t=\mathcal N(0,1)\) at every level, \(s_t(x)=-x\), the bracket vanishes and nothing moves — correct, since there is nothing to denoise.

Each operator individually maps one member of the family \(q_t\) to another. So does their composition:

\[ M_i \;=\; K_{t_i\to\hat t_i}\,\Phi_{\hat t_i\to t_{i+1}}, \qquad q_{t_i}M_i \;=\; q_{t_{i+1}}. \]

That single line is the entire foundation of what follows, so it is worth seeing rather than believing. Step through the widget: the histogram is the actual particle cloud, the smooth curve is the closed-form \(q_t\), and they are never fitted to each other.

Two things to take away. First, churn genuinely re-randomises: the reheat injects fresh noise, so duplicated particles separate again. That is the answer to the diversity problem from Step 1 — it is built into the sampler we already have. Second, and more important for what follows: the unmodified churn step is a transition that already does the right thing to the base distribution. Hold on to that. It is the reason the weight update is about to be so short.

Now put the two halves together. We want a cloud that, at every noise level, represents the tilted distribution

\[ \pi_t(x)\;\propto\; q_t(x)\,e^{\rho_t(x)},\qquad \rho_t(x)=\beta_t r_t(x). \]

A chain of these is called a bridge: a sequence of targets that starts somewhere easy (at \(t=1\) we set \(\beta_1=0\), so \(\pi_1=q_1\) is just noise and every particle has weight 1) and ends where we want (\(\pi_0\propto q_0e^{\beta_{\max}r}\)). Moving along it a little at a time is what avoids the collapse of Step 1.

Here is the point that trips people up, so let us be explicit about it. Applying a transition to a particle does not change its weight. Weights change only because we redefine what the cloud is supposed to represent. Push the cloud through \(M_i\) and carry the old weights along untouched, and what you now hold is the unnormalized measure

\[ \int q_{t_i}(x_i)\,e^{\rho_{t_i}(x_i)}\;M_i(x_i,dx_{i+1}), \]

the old tilted distribution diffused forward. That is a perfectly well-defined thing; it is simply not the thing we asked for, which was \(q_{t_{i+1}}e^{\rho_{t_{i+1}}}\). The weight update is the correction between the two. It is a retargeting step, not a consequence of moving.

And now the derivation, which is three lines. Multiply each particle by

\[ G_i(x_i,x_{i+1}) = \exp\bigl(\rho_{t_{i+1}}(x_{i+1})-\rho_{t_i}(x_i)\bigr) \]

and compute what the cloud represents. The factor \(e^{\rho_{t_i}(x_i)}\) we were carrying is cancelled by the \(e^{-\rho_{t_i}(x_i)}\) in \(G_i\), and the factor \(e^{\rho_{t_{i+1}}(x_{i+1})}\) depends only on the new state so it slides straight out of the integral over \(x_i\):

\[ \begin{aligned} \int q_{t_i}(x_i)\underbrace{e^{\rho_{t_i}(x_i)}}_{\text{old tilt}} M_i(x_i,dx_{i+1}) \underbrace{e^{\rho_{t_{i+1}}(x_{i+1})-\rho_{t_i}(x_i)}}_{G_i} &= e^{\rho_{t_{i+1}}(x_{i+1})}\int q_{t_i}(x_i)M_i(x_i,dx_{i+1})\\ &= q_{t_{i+1}}(x_{i+1})\,e^{\rho_{t_{i+1}}(x_{i+1})}. \end{aligned} \]

The last equality is the only place any property of the sampler is used, and the property used is exactly the one Step 2 established: \(q_{t_i}M_i=q_{t_{i+1}}\). The remaining integral does not care what \(M_i\) is made of — reheat, ODE, both, in any order — only that it carries the base marginal correctly. In log form:

The widget below runs it. Toggle the correction off and the weighted cloud does not merely get noisier — it converges to the wrong distribution, sitting on the untilted \(q_t\) while the target curve has moved left.

Before assembling anything, we have to clear up the thing that makes this construction feel like a trick. A particle gets pushed through a kernel — it moves in space, it moves in time, it lands somewhere with a different density — and its weight does not change. That looks wrong.

It looks wrong because of a natural but mistaken picture: that the weight is something the particle carries, like mass, which ought to be affected by where it goes. It is not. A weight is a ratio between two distributions, evaluated at the particle:

\[ w^n \;=\; \frac{\text{the distribution I want}}{\text{the distribution my sampler actually produces}}\Bigg|_{x^n}. \]

Nothing in that expression is a property of \(x^n\) alone. It is a statement about a mismatch between two recipes, and \(x^n\) is merely where we happen to be measuring it.

Now the punchline. Move every particle through a kernel \(M\). What happens to the two things in the ratio? Both get moved — the sampler's distribution becomes \(PM\), and the distribution we want, if we carry the weights along unchanged, becomes \(TM\). One line of algebra says so exactly. With \(x^n\sim P\) and \(w^n=(T/P)(x^n)\), draw \(y^n\sim M(x^n,\cdot)\):

\[ \mathbb E\Bigl[\textstyle\sum_n w^n f(y^n)\Bigr] = \int\!\!\int P(dx)\,\frac{T}{P}(x)\,M(x,dy)\,f(y) = \int\!\!\int T(dx)M(x,dy)f(y) = \int (TM)(dy)\,f(y). \]

So the weighted cloud, after moving, represents \(TM\) — the old target, carried forward by the very same kernel. The mismatch between numerator and denominator is unchanged because the same operation was applied to both. That is why moving is free.

And that is exactly our situation. We do not want \(TM\), the old tilted distribution smeared forward. We declared that we want \(\pi_{t_{i+1}}\propto q_{t_{i+1}}e^{\rho_{t_{i+1}}}\) — a fresh target at the new level. The weight update is the price of that redefinition, and of nothing else.

Here is where it gets easy rather than hard. The correction from \(TM\) to the new target looks like it should require knowing \(TM\), which is a smeared-out object nobody can write down. But we never compute it. Instead we recompute the whole weight from scratch, using the definition above:

\[ w_{i+1} \;=\; \frac{\text{want}}{\text{produce}} \;=\; \frac{q_{t_{i+1}}(x_{i+1})e^{\rho_{t_{i+1}}(x_{i+1})}}{\;q_{t_{i+1}}(x_{i+1})\;} \;=\; e^{\rho_{t_{i+1}}(x_{i+1})}. \]

The denominator is \(q_{t_{i+1}}\) — the law of the particle after the move — and it is knowable only because Step 2 established that the churn kernel carries \(q_{t_i}\) to \(q_{t_{i+1}}\). That single fact is what turns an intractable correction into a cancellation. And the base density \(q_{t_{i+1}}\), unknown though it is, appears in both numerator and denominator and drops out.

So the weight of a particle, at every level, is just the current tilt \(e^{\rho_t(x_t)}\). The "update" is nothing but the bookkeeping of a running product:

\[ \log w_{i+1} = \underbrace{\log w_i}_{\rho_{t_i}(x_i)} + \bigl(\rho_{t_{i+1}}(x_{i+1})-\rho_{t_i}(x_i)\bigr) = \rho_{t_{i+1}}(x_{i+1}). \]

It telescopes. Run the sampler with no resampling and the accumulated log-weight is identical to \(\rho_t\) evaluated at the particle's current position — to machine precision, not approximately. The increments are only needed because resampling periodically resets the weights to 1, and after a reset the running product has to restart from the current tilt.

The widget below shows all of it at once. Watch stage 2 in particular: the bars (the algorithm's cloud with unchanged weights) land on the orange curve — which is an independently simulated \(\pi_{t}M\), computed by drawing exact samples from \(\pi_{t}\) and pushing them through the same kernel. Two completely different computations agreeing is what "carrying weights unchanged means you represent \(TM\)" looks like in practice.

If you already know AIS or SMC

Then you already know this algorithm; it is the same template with one box filled in differently. Every sequential importance sampler has the form weight update, move, (optionally) resample, with the increment \(G_i\) chosen so the weighted cloud tracks the current target. The two settings differ only in what the move is asked to do.

In annealed importance sampling you build a ladder \(\gamma_0\to\gamma_K\) by hand, and you move with an MCMC kernel \(T_k\) that is invariant for \(\gamma_k\), i.e. \(\gamma_kT_k=\gamma_k\). Invariance is the AIS version of "moving is free": the kernel maps the target to itself, so the move costs no weight, and the entire increment comes from the target you changed by hand, \(\gamma_k(x)/\gamma_{k-1}(x)\) — two distributions, one point.

Here the move is not invariant, it is a transport: \(M_i\) maps \(q_{t_i}\) to a genuinely different distribution \(q_{t_{i+1}}\). That is a strictly stronger thing to have, and it does most of the work for you — the base part of the target is transported for free by the dynamics, and the weights are left to handle only the tilt. That is why the increment reads \(\rho_{t_{i+1}}(x_{i+1})-\rho_{t_i}(x_i)\): two tilts at two different points, because the particle genuinely moved and the target genuinely moved with it.

	AIS / SMC with MCMC moves	churn sampler + FK steering
the ladder	\(\gamma_k\), designed by hand	\(\pi_t=q_te^{\rho_t}\), indexed by noise level
the move	\(T_k\) invariant: \(\gamma_kT_k=\gamma_k\)	\(M_i\) transports: \(q_{t_i}M_i=q_{t_{i+1}}\)
what the move buys	mixing — decorrelates the cloud	mixing and the whole base marginal
weight increment	\(\dfrac{\gamma_k(x_{k-1})}{\gamma_{k-1}(x_{k-1})}\) — one point	\(e^{\rho_{t_{i+1}}(x_{i+1})-\rho_{t_i}(x_i)}\) — two points
where the score lives	nowhere	inside \(M_i\) only — never in the weight

Resampling then plays the identical role in both: it is optional, it does not change what the cloud represents, and it trades weight variance for duplicated particles. The difference is that a churn sampler repairs the duplication for free on the next step, because the reheat injects fresh noise into every particle.

Step 3's derivation goes through so quickly that it is worth stopping to ask what happened to the terms you would expect. Half of \(M_i\) is a deterministic map \(y=\Phi(x)\). Every time you push a density through a deterministic map, a Jacobian appears — that is the change-of-variables formula, and there are no exceptions to it. So where did it go?

It did appear. It appeared twice, and cancelled. Carry the weights unchanged through \(\Phi\) and the cloud represents, by change of variables,

\[ \underbrace{q_{\hat t}\bigl(\Phi^{-1}(y)\bigr)\bigl|\det D\Phi^{-1}(y)\bigr|}_{\text{this is } q_{t_{i+1}}(y)} \;\cdot\; e^{\rho_{\hat t}(\Phi^{-1}(y))}. \]

The Jacobian sits entirely inside the first group — and that group is \(q_{t_{i+1}}(y)\), because "\(\Phi\) transports \(q_{\hat t}\) to \(q_{t_{i+1}}\)" is precisely the statement that the density-with-Jacobian equals the new marginal. The base density we wanted is already there, Jacobian included. All that is left over is the wrong tilt factor, evaluated at the ancestor — and fixing that is the same endpoint subtraction as before, \(\Delta\log w_\Phi=\rho_{t_{i+1}}(y)-\rho_{\hat t}(x)\).

Said plainly: the Jacobian cancels between the proposal and the target because both are built on the same base flow. You are not tracking how \(\Phi\) distorts volume; you are relying on the fact that whatever distortion it produces is by definition the distortion that turns \(q_{\hat t}\) into \(q_{t_{i+1}}\).

The widget makes the cancelling quantity visible. Watch the gaps between tracer particles stretch and squeeze as they flow — that changing spacing is \(|\det D\Phi|\) — and then watch the transported density land exactly on the analytic \(q_{t_{i+1}}\) anyway.

Everything above rests on \(q_{t_i}M_i=q_{t_{i+1}}\), and that identity has a fragile ingredient: the ODE half only transports \(q_{\hat t}\) correctly if it is told it is starting at \(\hat t\). This is the one place where reordering the sampler creates a bug that is easy to write and hard to see.

The reasoning is short. After the reheat, \(\hat x_i\) is a sample from \(q_{\hat t_i}\) — a wider, smoother distribution than \(q_{t_i}\). The score network is a function of two arguments, and the argument that says "how noisy is this" must match the state you hand it:

\[ s_{\hat t_i}(\hat x_i) = \nabla_{\hat x_i}\log q_{\hat t_i}(\hat x_i). \]

Calling \(s_{t_i}(\hat x_i)\) instead asks the network to denoise a \(\hat t_i\)-noisy state as though it carried only \(t_i\) worth of noise. It will answer — score networks do not refuse — but it will under-remove noise, and the flow will not transport \(q_{\hat t}\) anywhere in particular. In code this is a one-token bug:

# inside one churn step
x_hat = forward_transition(x, t_curr, t_hat)   # exact; needs no score
v     = velocity(x_hat, t_hat)                # ✓ state and time agree
v     = velocity(x_hat, t_curr)               # ✗ the bug: stale time
x_next = heun_step(x_hat, v, t_hat, t_next)

Drag κ in the widget. The top panel shows the two score functions the two calls return; at κ=0 they are the same call and the curves coincide, and as the churn grows they peel apart. The bottom panel runs the full sampler both ways and measures the damage against the analytic \(q_t\).

So far the tilt has been one specific thing. In practice you will want to vary it — anneal its strength, make the reward itself depend on the noise level, evaluate it on the denoised state instead of the noisy one. It is worth stating the recipe in full generality, because it turns out to be completely insensitive to those choices, and because the one thing that does break it is easy to do by accident.

Set it up the way you would state it. You have \(q_t\), samples \(x_t\), and a kernel that advances \(x_t\) to \(x_{t+dt}\) with the property that \(x_{t+dt}\) is a sample from \(q_{t+dt}\). You want to bias both ends. The general object is a tilt function attached to each time index — call it \(\rho^{(i)}\), an arbitrary function of \(x\) chosen freely for each \(i\) — defining the target

\[ \pi^{(i)}(x)\;\propto\;q_{t_i}(x)\,e^{\rho^{(i)}(x)} . \]

Now apply Step 3 mechanically. The weight is always want-over-produce. After the move the particle's law is \(q_{t_{i+1}}\), so

\[ w_{i+1}=\frac{q_{t_{i+1}}(x_{i+1})e^{\rho^{(i+1)}(x_{i+1})}}{q_{t_{i+1}}(x_{i+1})}=e^{\rho^{(i+1)}(x_{i+1})}, \qquad \frac{w_{i+1}}{w_i}=e^{\rho^{(i+1)}(x_{i+1})-\rho^{(i)}(x_i)} . \]

Concretely, for the three cases you are likely to write:

you write	\(\Delta\log w_i\) is	what changed
\(\rho(x)=\beta\,r(x)\)	\(\beta\bigl[r(x_{i+1})-r(x_i)\bigr]\)	only the particle
\(\rho_t(x)=\beta_t\,r(x)\)	\(\beta_{t_{i+1}}r(x_{i+1})-\beta_{t_i}r(x_i)\)	particle and strength
\(\rho_t(x)=\beta_t\,r(x,t)\)	\(\beta_{t_{i+1}}r(x_{i+1},t_{i+1})-\beta_{t_i}r(x_i,t_i)\)	particle, strength, reward

Look at the first row, because it is the one that catches people. The reward does not depend on time at all, and yet the weight still changes. It changes because the particle moved, not because the reward moved. Step 3's ratio has \(x\) in it; move \(x\) and the ratio is a different number. A zero update would mean the target had not changed, and it has — \(\pi^{(i+1)}\propto q_{t_{i+1}}e^{\beta r}\) is a different distribution from \(\pi^{(i)}\propto q_{t_i}e^{\beta r}\), because \(q\) moved underneath.

Your question about whether to use \(t\) or \(t+dt\) now has a precise answer, and it is a slightly surprising one. Feeding the new state to the old tilt function is not an error — it just means you have declared \(\rho^{(i+1)}:=\rho_{t_i}\), i.e. shifted your schedule by one index. Any assignment of functions to indices is a legal Feynman–Kac model. What is not optional is consistency: whatever function you use for the new endpoint of step \(i\) must be the same function you use for the old endpoint of step \(i+1\). Get that right and the sum telescopes:

\[ \log w_N=\log w_0+\sum_{i=0}^{N-1}\bigl(\rho^{(i+1)}(x_{i+1})-\rho^{(i)}(x_i)\bigr)=\rho^{(N)}(x_N). \]

Read that off carefully, because it is the practical payoff of the whole step. The terminal weight depends only on the last tilt function. Every intermediate \(\rho^{(i)}\) cancels. So the annealing schedule cannot bias your endpoint — it is a pure variance knob, exactly like the proposal twist in Step 8. Choose it to keep the ESS healthy; it has no say in what you converge to.

This is sharper than a statistical statement. Since the particle trajectories do not depend on \(\beta_t\) at all — the schedule touches only the weights — and the telescoped final weight is \(\rho^{(N)}(x_N)\) regardless of the route, two runs with different schedules and the same seed end with numerically identical weights. Not the same distribution: the same numbers. Drag the schedule shape below and the terminal histogram does not move by a pixel, while the ESS trace reshapes completely. Then flip the consistency switch to see the one thing that genuinely breaks.

The endpoint weight is exact, but Step 1's problem has not gone away, only been spread out: particles still wander wherever the base model takes them, and the weights still do all the steering. The natural improvement is to aim the randomness — bias the churn noise toward high reward — and then correct for having done so.

This is clean to do because the churn kernel is a Gaussian whose density we know: \(K_i(\hat x\mid x_i)=\mathcal N(\hat x; m_i, V_i)\). Sample instead from any \(\widetilde K_i\) with a known density, and importance sampling supplies the exact fix — a likelihood ratio:

\[ \Delta\log w_i=\rho_{t_{i+1}}(x_{i+1})-\rho_{t_i}(x_i) + \log K_i(\hat x_i\mid x_i)-\log\widetilde K_i(\hat x_i\mid x_i). \]

Which \(\widetilde K_i\) though? Ideally we would sample from \(\mathcal N(\hat x;m_i,V_i)\,e^{\rho_{\hat t}(\hat x)}\), but that is not a Gaussian for a general reward. So linearize: approximate \(\rho_{\hat t}(\hat x)\approx \rho_{\hat t}(m_i)+a_i^\top(\hat x-m_i)\) with \(a_i=\nabla\rho_{\hat t}(m_i)\). Now the product is Gaussian, and completing the square in the exponent, \(-\tfrac12(\hat x-m)^\top V^{-1}(\hat x-m)+a^\top(\hat x-m)\), shifts the mean by \(Va\) and leaves the covariance alone:

\[ \widetilde K_i=\mathcal N\bigl(m_i+V_ia_i,\;V_i\bigr). \]

Nudge the mean uphill on the reward, by an amount proportional to how much noise this step was going to inject anyway. The correction is then a two-term formula you can write down by hand — subtract the two Gaussian log-densities and the quadratic terms cancel:

\[ \log K_i-\log\widetilde K_i = -a_i^\top(\hat x_i-m_i)+\tfrac12 a_i^\top V_ia_i. \]

Note what is not being twisted: the deterministic ODE. Bending a deterministic flow gives you two maps whose transition kernels are mutually singular — both put all their mass on a measure-zero set, so the likelihood ratio between them does not exist as an ordinary density ratio, and you are back to tracking inverse maps and Jacobians. Twisting a Gaussian is easy precisely because Gaussians have densities. Twist the stochastic half; leave the deterministic half alone.

Now drag the twist strength, and watch two things that behave differently.

The weighted mean sits on the target for every twist strength — as it must, since the likelihood ratio is exact for any proposal. But the ESS traces a hump: it climbs steeply from about 9% untwisted to around 60% near c≈0.35, then falls off a cliff, ending below the untwisted sampler at c=1.

The diagnostic bar explains the cliff. The twist displaces the proposal mean by \(V_ia_i\); the noise it was going to add has scale \(\sqrt{V_i}\). Once the displacement is a large multiple of \(\sqrt{V_i}\), the proposal and the base kernel barely overlap, and the ratio \(K_i/\widetilde K_i\) becomes a wildly varying number — the very thing we introduced the twist to avoid. The linearization was only ever valid near \(m_i\), and at c=1 the reward here is steep enough to throw the proposal far outside that neighbourhood.

This is the honest shape of the technique. Twisting is a variance knob, not a correctness knob — and like all variance knobs it has a setting past which it does harm. The correction keeps you exact the whole way; it cannot keep you useful the whole way. Note the practical reading of the failure: at c=1 the estimate is unbiased in the mathematical sense but is being carried by a handful of effective particles, so any single finite run is unreliable — the number can be wrong even though the estimator is right.