Test-Time Generation

Zefan Cai¹   Qinzhe Hu¹   Yuchen Zhu²   Hao Tan³   Tianyuan Zhang⁴   Sai Bi³   Haoyi Qiu⁵   Yicong Hong³   Haozhe Zhao⁶   Jason Kuen³   Haoliang Wang³   Ruiyi Zhang⁷   Ryan A. Rossi³   Wanrong Zhu⁸   Molei Tao²   Wen Xiao⁹   Junjie Hu¹   Jiuxiang Gu³

¹University of Wisconsin–Madison   ²Georgia Institute of Technology   ³Adobe   ⁴Massachusetts Institute of Technology   ⁵University of California, Los Angeles   ⁶University of Illinois at Urbana-Champaign   ⁷Apple   ⁸Meta   ⁹Microsoft

From regression to generative recall

A fast-weight memory compresses the context into a fixed-size state h that is written online as tokens or views arrive. Existing designs such as LaCT read it by deterministic regression, whose squared-loss optimum is the conditional mean: when a query is compatible with several stored values, a point prediction can merge them even if the state still holds the alternatives. TTG keeps LaCT's memory architecture and chunked update operator and changes how the memory is read, not how large it is.

On a write, each observed value is corrupted with sampled noise and the chunk update trains the fast-weight MLP as a key-conditioned denoiser; the slow parameters stay fixed at test time. On a read, each query starts from fresh noise and follows a query-conditioned denoising trajectory with the state frozen. More readout steps add fast-MLP evaluations, not state. In language modeling each chunk reads only the preceding context before its own update, which keeps the model causal; in novel view synthesis the observed posed images update the scene state before target-camera queries read it.

Language modeling

Long-context language modeling, evaluated up to 32K. PTL is the per-token loss at positions 30K–32K of 32K-token Book-3 sequences, as stored by each run's evaluation; RULER is accuracy (%) over 13 tasks, averaged over 4K, 8K, 16K and 32K. TTG uses one write repeat and one trained readout step (r=d=1). Scale labels name model families; TTG's conditioning changes the total parameter count. Bold marks the best value in each column among the bounded-state models; the full-attention Transformer† and YOCO† keep a key–value cache that grows with context and are excluded from that comparison.

Show RULER by length

TTG adds 5.02, 1.88 and 3.8 mean RULER points over LaCT at 124M, 760M and 3B, and 3.78, 5.44 and 5.6 points at 32K. TTG leads LaCT at every length at 124M and 3B; at 760M it leads at 8K–32K and trails at 4K (43.96 vs 46.67). At 124M it also exceeds full attention on mean RULER (17.38 vs 16.52). Loss does not follow retrieval: at 3B TTG has 0.039 higher PTL than LaCT, and KDA-SWA has the lowest bounded-state PTL at 124M and 3B. One readout step keeps the gain; on a checkpoint trained with one step, extra evaluation steps made PTL worse, not better.

Novel view synthesis

An LVSM-style model with an 8-layer, width-512 backbone writes the observed posed views into the scene memory and generates target-view features from it. Object-level models train on Objaverse and are tested on all 1,019 GSO objects; scene-level models train and test on DL3DV-10K, with 140 test scenes. TTG reaches 24.93 dB on GSO and 16.59 dB on DL3DV, 0.94 dB and 0.45 dB above LaCT, and it also beats DeltaNet and Gated DeltaNet on both sets. The growing-cache Transformer† stays higher, at 25.36 and 17.41 dB. PSNR is an L2 measure and cannot by itself show whether a readout avoids averaging; the retrieval and two-value tests are sharper.

PSNR in dB, averaged over all test scenes and all 23 next-view positions, with one generated sample. Bold marks the best bounded-state value in each column; † growing-cache reference. TTG, DeltaNet and Gated DeltaNet share seed-0 view sequences; the archived LaCT and Transformer results use seed 9595, so those comparisons are not paired by view sampling.

Selected reconstructions

Can one query recover two stored values?

A controlled probe writes two values per key into the same gated-MLP fast-weight memory, so N keys hold 2N bindings; the values lie on the unit sphere with separation 1.732. Coverage is the fraction of keys for which 32 reads include a sample strictly closer than half the separation to each stored value. TTG reaches 65.9% at N=4 with S=4 readout steps and 34.4% at N=16 with S=8.

The deterministic controls are LaCT, LaCT with four times the fast weights at matched slow-parameter count, and a matched-noise control that keeps TTG's loss, conditioning, widths and denoising loop but replaces the noise with a learned constant. All three have zero two-value coverage by construction, and at N=64 the larger LaCT only moves its output closer to the two-value mean. Freezing the readout noise on trained TTG weights also removes the coverage, while a one-value-per-key control succeeds in both families. The controls show that stochastic reads reach both values; they do not isolate denoising training as the only cause.

Coverage needs only one hit near each value among 32 draws, so it can rise without better average samples: at N=16 it goes from 0 to 34.4% while the mixture NLL barely moves (70.8 to 69.9). At N=64 every arm has zero coverage, and at S=4 TTG's mixture NLL is about 3.5× LaCT's: at the highest tested load, generative reading does not overcome write-side interference.

What the results establish

TTG changes how a fixed-size fast-weight memory is read, not its size. In the reported settings it improves mean RULER over LaCT from 124M to 3B and next-view PSNR over the tested fast-weight baselines, and the two-value probe links competing-value recovery to stochastic reading at low and moderate load. The language-model gain holds with one readout step, so it may come from the learned denoising readout rather than extra iterations; that result does not show recovery of multiple values, and loss and retrieval rank the models differently.

Extra noisy writes and readout steps cost compute and do not help uniformly: r=d=1 is a low-cost default, not a universal optimum, and the depth sweep suggests evaluation depth should match training depth. The LaCT and Transformer comparisons in novel view synthesis are not paired by view sampling; identical NVS reruns differ by a median 0.035 dB and at most 0.165 dB, a spread rather than a confidence interval. At the highest probe load, write-side interference limits every readout.

Citation

@misc{cai2026testtimegeneration,
  title={Test-Time Generation},
  author={Zefan Cai and Qinzhe Hu and Yuchen Zhu and Hao Tan and Tianyuan Zhang and Sai Bi and Haoyi Qiu and Yicong Hong and Haozhe Zhao and Jason Kuen and Haoliang Wang and Ruiyi Zhang and Ryan A. Rossi and Wanrong Zhu and Molei Tao and Wen Xiao and Junjie Hu and Jiuxiang Gu},
  year={2026},
  url={https://zefan-cai.github.io/TTG.github.io/},
}

Copy BibTeX