The Problem问题
A ternary model stores each weight as −1, 0 or +1 times a per-row scale, so post-training it is a large-scale discrete optimization problem: which weights should change, and to which value? The standard approach keeps a hidden full-precision copy of every weight, trains it through a straight-through estimator, and rounds it in each forward pass. That copy and its optimizer state take far more memory than the 2-bit model itself, and most small updates never change a deployed weight. 三值模型的每个权重只能取 −1、0 或 +1，再乘一个逐行的 scale，所以它的后训练是一个大规模离散优化问题：该改哪些权重，改成哪个值？常规做法是给每个权重维护一份隐藏的全精度副本，用 straight-through estimator 训练，前向时再取整。这份副本和优化器状态占的内存远大于 2-bit 的模型本身，而且大部分微小更新改变不了部署的权重。
The Idea想法
I am exploring zeroth-order search directly in the deployed format: candidate code edits are scored by forward rollouts and task rewards alone, with no full-precision copy and no backpropagation through the model. 我想探索直接在部署格式上做零阶搜索：候选的码修改只靠前向 rollout 和任务 reward 打分，不需要全精度副本，也不需要对模型反传。
Open Questions开放问题
Three questions are open. Which parts of the network deserve the search budget? How many codes should change at each step? How can we confirm that an edit is a real improvement and not rollout noise? The goal is to reach the same quality at a lower total rollout cost than evolution strategies on quantized weights. 现在有三个开放问题：搜索预算该分给网络的哪些部分？每一步该改多少个码？怎么确认一次修改是真的变好，而不是 rollout 噪声？目标是在同样的质量下，总 rollout 成本低于量化权重上的进化策略。
Key Papers关键论文
- BitNet b1.58 2B4T Technical Report (Ma et al., 2025) — the ternary LLM that made this format practical. — 让三值大模型走向实用的代表工作。
- GXNOR-Net (Deng et al., Neural Networks 2018) — direct transitions between ternary states, without full-precision memory. — 不用全精度内存，直接在三值状态之间转移。
- Latent Weights Do Not Exist (Helwegen et al., NeurIPS 2019) — Bop flips binary weights directly, with no latent weights. — Bop 不用 latent 权重，直接翻转二值权重。
- PV-Tuning (Malinovskii et al., NeurIPS 2024) — alternates discrete and continuous updates for LLM compression. — 在大模型压缩中交替做离散更新和连续更新。
- Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity (Guo et al., 2024) — Fisher-informed coordinate selection for sparse zeroth-order fine-tuning. — 用 Fisher 选坐标，做稀疏的零阶微调。
- Quantized Evolution Strategies (Xu et al., 2026) — reward-based search in quantized weight space, and the direct baseline here. — 在量化权重空间上用 reward 搜索，是这里最直接的 baseline。