The router
Nine tools reach the model out of however many exist. What decides which nine, what the 48 KiB classifier inside it does, and why the same model runs on a microcontroller.
Nine tools, not all of them
A prompt carrying every tool on the machine is a worse prompt: a small model picks worse from a long list, and every unused description is context spent. So the router ranks the catalog for each message and shows the model the top nine. If the tool you expected is not among those nine, the model never had the chance, and no amount of prompt work will fix it.
tacet why "hangi dosyalar değişti"
tacet bench check my-tools.json --portabletacet why prints the ranking without loading any weights, so it costs nothing to ask. When a case goes red it is the first thing to run: it tells you whether to blame the routing or the model.
A trigger list, and then a small model
The base of the router is a hand-written trigger table — substrings and weights, readable and editable. It is good at what it covers and structurally blind to what it does not: a request that names no keyword reaches nothing.
So a 48 KiB int8 classifier now ships inside the router as a second signal. On the two extraction tools, the trigger list reaches 87 of the 105 requests that expect one; with the classifier head that becomes 102. It catches 15 of the 18 requests no substring can reach, including the ones that name no place at all.
The head is wired as a signal that adds, so a bad prediction cannot demote a tool the trigger list found. That matters because an earlier version of the same head, trained without the other tools’ work as negatives, called 38% of the other suites’ messages an extraction request and cost fourteen top-three positions. It is in the repository with its false-positive rate measured rather than asserted.
Down to 92 KiB
A field with five legal values is not a generation problem. It is an argmax over a handful of classes, where the guarantee the automaton buys is free because there is nothing to emit from. So the same job was written as hashed character n-grams into one int8 weight per class, trained on generated examples and scored on 131 human-written cases — 95 of them written after the model was trained.
| SmolLM2-135M | Classifier | |
|---|---|---|
| Size | 528 MiB resident | 92 KiB |
| Work per message | ~200 tokens generated | 4,266 integer ops |
| search_filter tool | 4/5 | 5/5 |
| search_filter slots | 1/5 | 15/15 |
| message_intent intent | 0/4 | 3/4 |
Which is what makes an ESP32-S3 a real target rather than a slide. A decode step reads every weight once, so tokens per second cannot exceed bandwidth divided by size, and that board’s PSRAM sustains about 40 MB/s: a 135M model at Q4 is 68 MB and cannot beat 0.59 tokens per second even if it fitted, which it does not. At 92 KiB the weights are 18% of the internal SRAM and the bandwidth wall never applies — 4,266 ops is 44.4 µs at 240 MHz.
What it cannot do is the honest half
Three of the fields are open text — copying a span, not choosing a class — and stay with the host. Nine cases is a small denominator. And the device figures are arithmetic from a measured operation count, not silicon: nothing has been run on a board.
The trainer and the C implementation are compared accumulator by accumulator rather than answer by answer, because comparing answers was not enough: breaking the letter folding changed every Turkish message’s features and flipped no prediction at all. The tighter check found two real bugs in a minute — Python lowercases İ into two codepoints, and reading every non-ASCII character as two bytes mistakes an em dash for a Turkish letter.