这是旧页,不再更新。回首页 · Old page, no longer updated.
21 NAND
Driver
TapeOut · BNB Chain
impossible LLM · v1 存档archive

这是 v1 存档(2026-09-15,54,147 门,贪心、只用维基百科)。它会掉进 “the 2008 , the 2008” 这样的吸收环。现行版本见 impossible LLM v2。This is the v1 archive (2026-09-15, 54,147 gates, greedy, Wikipedia only). It falls into absorbing loops like “the 2008 , the 2008”. The current version is impossible LLM v2.

impossible LLM v1 — 把 MiniCPM5-2B 转写成一张 NAND 网表MiniCPM5-2B transcribed into a NAND netlist

一张 54,147 个 NAND 的组合网表,读 6 个 MiniCPM token、一次 step() 输出下一个 token。逐门求值在这一页的 Web Worker 里跑,没有后端、没有 API key。A combinational netlist of 54,147 NAND gates reads 6 MiniCPM tokens and outputs the next token in a single step(). Gate-by-gate evaluation runs in a Web Worker on this page — no backend, no API key.

续写Continuation

加载分词器中…(gzip 后约 2 MB)Loading tokenizer (~2 MB gzipped)…

建议试试:The history of the Roman Empire —— 点生成,看网表逐 token 写下去。Try: The history of the Roman Empire — press Generate and watch the netlist write on, token by token.

模型Model

门数Gates54,147 NAND · 0 LATCH
接口Interface102 in(6 × 17 位 token id,最老在低位)/ 13 out(top-8192 码)102 in (6 × 17-bit token ids, oldest in the low bits) / 13 out (top-8192 code)
蒸馏DistillationMiniCPM5-2B BF16,argmax,1024-token 预热,16M 标注MiniCPM5-2B BF16, argmax, 1024-token warm-up, 16M labels
一致率(同体裁)Agreement (same genre)约 0.23 · 测试窗口与训练区同一语料体裁(基线约 0.07)about 0.23 · test window in the same corpus genre as the training range (baseline about 0.07)
一致率(跨体裁)Agreement (cross-genre)约 0.11 · 测试窗口落在语料另一段体裁上(基线约 0.06)。这才是它面对陌生文本时的诚实数字。about 0.11 · test window in a different genre of the corpus (baseline about 0.06). This is the honest number for text it hasn't seen.
教师Teacher同两个窗口上都是 约 0.49 —— 教师几乎不掉,所以掉的不是数据质量,是「背下来的表不泛化」about 0.49 on both windows — the teacher barely drops, so the drop is not about data quality but about “a memorized table doesn't generalize”
测量精度Measurement precision同体裁内多个不相交 20 万 token 窗口的散布约 ±0.01;跨体裁比较必须说明体裁,否则散布看起来像 ±0.03(那是把两种体裁混在一起的假象)。小于 0.01 的差别不显著。Several disjoint 200K-token windows within one genre scatter by about ±0.01; cross-genre comparisons must state the genre, otherwise the scatter looks like ±0.03 (an artifact of mixing two genres). Differences below 0.01 are not significant.
天花板Ceiling全表 0.339 / K=1M 截断 0.35186(均为近分布)· top-8192 覆盖 0.92225full table 0.339 / K=1M truncation 0.35186 (both near-distribution) · top-8192 coverage 0.92225
每步 gasGas per step133,993,004(1.91 个块,建模)133,993,004 (1.91 blocks, modeled)
流片Tape-out尚未流片 —— 12 片、实测 estimateGas 174,787,056、合计 0.011139 BNBnot taped out — 12 chunks, measured estimateGas 174,787,056, total 0.011139 BNB
已知缺陷Known defect这一版的查表键是用一个有缺陷的哈希算的:numpy 的 uint64 移位超过 63 位返回 0,导致第 5、6 个 token 完全没进键、第 4 个只进了 17 位中的 13 位。所以尽管页面确实把 102 位喂进网表,深层的表实际上只区分约 3.76 个 token(铁证:W=6/5/4 三层的不同键数是 11,573,526 / 11,573,527 / 11,573,528,彼此差不到 2)。下面那些一致率是这张网表实测的,所以数字本身没错;修正键后的版本在部署体裁上约 0.214、满表天花板 0.288,正在重造,届时替换本页。The lookup keys in this version were computed with a buggy hash: numpy uint64 shifts of more than 63 bits return 0, so the 5th and 6th tokens never made it into the key and the 4th contributed only 13 of its 17 bits. So although the page really does feed 102 bits into the netlist, the deep tables effectively distinguish only about 3.76 tokens (hard evidence: the W=6/5/4 levels have 11,573,526 / 11,573,527 / 11,573,528 distinct keys, differing by less than 2). The agreement rates here are measured on this netlist, so the numbers themselves are correct; a version with fixed keys scores about 0.214 on the deployment genre with a full-table ceiling of 0.288, is being rebuilt, and will replace this page.

链上对照On-chain check

这一页暂时没有链上逐 token 对照——网表还没烧。已流片的 cid 285 / 286 / 287 有活的 eth_call 对照,见存档页 一步模型阶梯和逐字续写演示(两页已下线,保留为工程存档)。烧录后这一页会补上同样的一栏。This page has no on-chain token-by-token check yet — the netlist has not been burned. The taped-out cid 285 / 286 / 287 have live eth_call checks; see the archive pages one-step model ladder and character-by-character continuation demo (both retired and kept as engineering archives). The same section will be added here once it is burned.

范围(说清它不做什么)Scope (what it does not do)

  • 它不是 MiniCPM5-2B。近分布一致 23.5%,远分布只有 11.0%,而教师在两个窗口上都是 49.4%。这是转写,不是等价。It is not MiniCPM5-2B. Near-distribution agreement is 23.5% and far-distribution only 11.0%, while the teacher scores 49.4% on both windows. This is a transcription, not an equivalent.
  • 它背下来的东西不泛化。同一个模型在同体裁窗口上约 0.23、跨体裁窗口上约 0.11,而教师在两者上都是约 0.49,分毫不差。教师的权重能迁移,一张记住的表不能。What it memorized does not generalize. The same model scores about 0.23 on same-genre windows and about 0.11 on cross-genre windows, while the teacher scores about 0.49 on both, without budging. The teacher's weights transfer; a memorized table does not.
  • 它不会「变旧」,但会遇到「不一样的文本」。受控实验里,同一个模型在离训练区 16M 和 48M token 的窗口上得分相同,而在反方向同样距离的窗口上高得多——距离是对称的,得分不是。所以下降不是时间造成的老化,而是测试文本换了体裁:语料在某个位置有一道明显的成分分界(分界后每段的不同标签多 23%,落在最高频 token 上的质量少 19%),教师在分界两侧一样准,变的是「能不能靠背」(最高频 1,000 个键的命中率 0.67 → 0.50)。烧上去的网表不会因为时间流逝而退化,但会因为遇到另一种文本而掉一档。It does not “go stale”, but it does meet “different text”. In a controlled experiment the same model scored the same on windows 16M and 48M tokens away from the training range, but much higher on windows the same distance away in the opposite direction — the distance is symmetric, the score is not. So the drop is not aging over time but a change of genre in the test text: the corpus has a sharp compositional boundary at one point (past it, each segment has 23% more distinct labels and 19% less mass on the most frequent tokens). The teacher is equally accurate on both sides; what changes is whether memorization works (hit rate of the 1,000 most frequent keys 0.67 → 0.50). A burned netlist will not degrade as time passes, but it drops a notch when it meets a different kind of text.
  • 训练数据选哪一段比多花门数更值钱。同样的门数,用最近的 1,600 万 token 训练而不是最早的,一致率从约 0.11 提到约 0.17;而且最近区间的单层 unigram 就打赢了最早区间的完整六层级联。Which slice of training data you choose is worth more than extra gates. At the same gate count, training on the most recent 16M tokens instead of the earliest raises agreement from about 0.11 to about 0.17; and a single-level unigram on the recent range already beats the full six-level cascade on the earliest range.
  • 只看 6 个 token,无状态、无记忆。更长的上下文买不起:门数随表项线性(约 6.7 NAND/项),5 个块/token 的预算只够约 15 万门。Sees only 6 tokens; stateless, no memory. Longer context is unaffordable: gates grow linearly with entries (about 6.7 NAND per entry), and a budget of 5 blocks per token buys only about 150K gates.
  • 只有 8,192 个可能的输出(教师 argmax 质量的 92.2%);表外的目标它永远说不出来。Only 8,192 possible outputs (92.2% of the teacher's argmax mass); targets outside that list can never be produced.
  • 不会聊天、不会调用工具。它是一个下一 token 预测器,不是 agent。Can't chat or call tools. It is a next-token predictor, not an agent.
  • 它会循环。转写的是教师的 argmax,没有采样随机位,所以是确定性的——确定性的贪心续写必然进入循环(试试上面那句,几个 token 后就开始重复)。这是设计的直接后果,不是缺陷修不掉:加采样位要多花门数,而 v0.2 采样器制造过一个换行吸收态。It loops. It transcribes the teacher's argmax with no random sampling bits, so it is deterministic — and deterministic greedy continuation inevitably ends up in a loop (try the sentence above; it starts repeating after a few tokens). This is a direct consequence of the design rather than an unfixable bug: sampling bits cost extra gates, and the v0.2 sampler once created a newline absorbing state.
  • 一致率之所以停在 0.23,不是门数不够,而是 6 个 token 的上下文里没有足够信息:在真正重复出现的上下文上,教师 argmax 的条件熵仍有 1.9–3.7 比特。Agreement stalls at 0.23 not for lack of gates but because 6 tokens of context do not carry enough information: on contexts that genuinely recur, the conditional entropy of the teacher's argmax is still 1.9–3.7 bits.