21 NAND Docs 中文

A tiny language model

impossible LLM v2: one netlist of 5,142,875 NAND gates. It reads the last 6 tokens and writes the next token in one evaluation.

How to use it

  1. Open the page and wait for it to load the netlist (7.9 MB compressed, 36 MB unpacked) and the tokenizer (about 2 MB).
  2. Type a beginning in the input box and click "Generate 48 tokens".
  3. Tick "Greedy" to fix the random bits at 0. The same beginning then writes the same text every time.

Everything is computed gate by gate in the browser. There is no backend.

How it works

For each token it writes: it takes the numbers of the last 6 tokens (102 bits) plus 4 random bits, runs them through 5,142,875 gates, and gets a 13-bit code, which is one of 8,192 common tokens.

Some numbers

ItemValue
Gates5,142,875 NAND gates, 0 latches
Input / output106 bits / 13 bits
TeacherMiniCPM5-2B, taking its most likely token
Training text10 million tokens of English Wikipedia, 8 million of web pages, 8 million of dialogue, 6 million of Chinese Wikipedia
Table entries293,717

How often it matches the teacher

Test textv1v2 greedyv2 sampled
English Wikipedia0.2350.3090.233
Dialogue0.0740.3030.226
Web pages0.1160.2410.177
Chinese Wikipedia0.0040.2190.174

The teacher itself scores about 0.49 on Wikipedia (measured at v1). With 80 beginnings, each writing 48 tokens: v1 got stuck in short loops every time; v2 sampled, 0%.

How it was checked

What it can't do

Next