The Little Robot That Lives on Your Mac

This is the story of a project with two acts. Act one: get an AI coding assistant running entirely on a MacBook — no cloud, no API keys, no one watching. Act two: the really interesting part — put that little AI in a loop, give it a goal and a score, and discover that you've accidentally built an optimizer… and an optimizer will cheat you blind unless you build the game so cheating doesn't pay.

1.A robot intern made of three parts

Forget "AI" for a second. What lives in this repo is a stack of three ordinary programs, sitting on top of each other on one MacBook:

Here's the division of labor, and it matters for everything that follows:

• The model (gemma4:e4b) is a frozen file of numbers. It can't touch your files. It can't run commands. All it does is read text and produce more text. That's it. That's all any language model does.
• Ollama is a waiter. It takes requests in a standard format (the same one OpenAI's cloud uses), carries them to the model, and carries answers back. Because it speaks the standard dialect, any tool built for cloud AI can be pointed at your laptop instead.
• pi is the agent — a small program that gives the model hands. It sends the model your request plus a menu of tools ("you may read files, write files, run bash"). When the model replies "I'd like to run wc -l README.md", pi actually runs it and feeds the result back. Around and around, until the job is done.

2.Why a "small" model isn't small (the restaurant trick)

The model chosen here, gemma4:e4b, is a Mixture of Experts (MoE) model. Here's the idea, no math required.

Imagine a restaurant with a huge kitchen: a sauce specialist, a grill specialist, a pastry chef, a fish guy — dozens of experts. When an order comes in, you don't wake up the whole kitchen. A head waiter glances at the ticket and routes it to the two or three specialists who matter for that dish. The kitchen is enormous; the work done per order is small.

MoE models are built the same way. The full model is big on disk (9.6 GB here), but for each word it generates, a little routing mechanism activates only a fraction of it — roughly 4 billion "active" parameters per token out of a much larger total. You get the knowledge of a big model at close to the speed of a small one. That's why it feels fast on an M4 Pro with 24 GB of memory.

3.The difference between describing work and doing work

This is the single most important lesson from act one, and almost nobody talks about it.

When pi offers the model its tool menu, the model is supposed to reply in a strict, machine-readable format — a tool call — something pi can parse and execute. But many models, especially small ones, do something subtly useless instead: they reply with a nice markdown snippet that describes the command. To a human reading the chat it looks identical. To pi, one is an instruction and the other is decoration.

# What a useless model says (Gemma 3 did this):
"Sure! Let's count the lines:
```bash
wc -l README.md
```"
        ← just words. Nothing happens.

# What a working model emits (Gemma 4 does this):
{"tool": "bash", "arguments": {"command": "wc -l README.md"}}
        ← pi parses this, RUNS it, returns the output.

Gemma 3 was tried first and failed exactly this way. Gemma 4 has function-calling trained directly into its weights — it learned, during training, that when given a tool menu, the right move is to emit the structured format. This is also why the project uses Ollama instead of MLX, even though MLX is ~30% faster at raw token generation: Ollama's per-model parsers reliably catch Gemma's tool calls; the MLX server didn't.

"Agent work is bottlenecked on tool execution, not on decode speed."

That line from the README is the engineering judgment of the whole act: a model that's 30% slower but actually does things beats a faster model that only talks about doing things. Speed of useless output is zero.

4.The silent killer: a window that's too small

One more concept and you'll have caught up to the project's hardest-won bug. A language model has a context window — the maximum amount of text it can hold in mind at once. Think of it as the model's desk. Everything it needs — its instructions, the tool menu, your request, the code it's working on — has to fit on the desk simultaneously.

Here's the trap that cost this project real debugging hours (it's Failure #7 in the build log): Ollama doesn't necessarily use the model's full window. Unless a desk size (num_ctx) is baked into the model configuration, Ollama defaults to a tiny one — about 4,000 tokens — and silently throws away whatever doesn't fit.

Why is this so nasty? Because the symptom is an empty response with a clean exit code. Every instinct says "my script is broken," and you go debug the wrong layer. The fix was to bake a bigger desk into a custom model variant — gemma-agent = gemma4:e4b + num_ctx 16384 + low temperature — and the very next cycle, everything worked.

5.Act two: the agent in a loop is an optimizer

So you have a little local agent that works. Now the ambitious idea: stop chatting with it and make it work autonomously. Give it a goal, a way to score itself, and let it grind: try something, measure, try again. That's the lfd/ directory — Loss-Function Development, a framework created by Elvis Sun. Half of this project is the question: does his meta-framework survive when the agent in the loop is a small local model instead of a frontier one?

The name comes from machine learning, where the "loss function" is the number you're trying to drive down. LFD's founding insight is worth engraving somewhere:

An agent in a loop is an optimizer, and every cheap path you don't fence off is a direction it will sprint down.

You've met this phenomenon even if you've never met an AI. Pay a factory per nail and you get thousands of tiny nails. Pay by the kilogram and you get one giant nail. Goodhart's law: when a measure becomes a target, it stops being a good measure. An optimizing agent doesn't share your intent — it shares your metric. Anything that raises the number is, to the optimizer, correct behavior. Lying, hardcoding answers, editing the grading script: all fair game unless structurally prevented.

The repo even keeps a cheat museum — a catalog of twelve real ways agents have "won" without solving anything. A taste:

That last one deserves a pause: even your anti-cheat system is an information channel, and a patient optimizer will mine it. The first draft of this very harness had that hole; red-teaming found it.

6.The big idea: move the intelligence into the harness

The original LFD method assumes a frontier-class model — something smart enough to read a protocol document and faithfully follow it: score yourself honestly, commit checkpoints, notice when you're stuck, stop on budget. A 7-billion-parameter local model cannot be trusted to do any of that. So this project makes one ruthless adaptation, and it's the best idea in the whole repo:

The model self-polices nothing. Every rule it could break is enforced in bash, outside the model.

The loop (loop.sh, ~250 lines of shell) owns scoring, checkpointing, stall detection, budgets, read-only enforcement, and stop conditions. The model's job per cycle shrinks to almost nothing: write down a hypothesis, make one change. Here's one full cycle:

Two refinements in that loop are easy to miss and very clever:

• Stage 0 vs. the outer loop. "Try a different approach!" is great advice for a working program stuck at a plateau — and terrible advice for a program that doesn't run yet. The harness distinguishes the two: while the score is 0, the directive is converge — fix the exact error shown, change nothing else. Only after a working baseline exists does it start injecting exploration. (This was learned the hard way: the early stall rule kept shoving the model away from fixing its own crash.)
• Feedback resolution matched to the model. A frontier model can debug from a bare "SCORE: 0.000." A 7B model can't — it needs the failure's shape. So the scorer emits up to two MISS: lines showing the input and the model's own wrong output or error — never the expected answer (that would leak the answer key). Enough traction to climb; nothing to memorize.

7.How do you stop a robot from cheating?

You can't make cheating impossible — the agent has unrestricted bash, so any file on the machine is technically reachable. The honest threat model in the README says it plainly: these are fences with tripwires, not walls. The design goal is to make every cheat either unprofitable or detectable-and-score-voiding. Three mechanisms do most of the work:

Fence 1: the train/test split, weaponized

The agent practices against 25 dev cases and sees its dev score every cycle. But passing dev means nothing. Acceptance is measured on a holdout set of 15 cases the agent has never seen — and it can't even run the holdout scorer, because holdout scoring demands a secret key that only loop.sh holds, is rate-limited to 3 calls per run, and is logged to an audit file. This is exactly why schools keep the final exam in a locked drawer and only hand out practice problems.

Fence 2: the canary

A canary is a unique nonsense string planted inside the vault. If that string ever shows up in the agent's workspace, there is exactly one explanation: the agent went rummaging in the vault. The lint greps for it every scoring pass — match means VOID. It's the bait car of this codebase.

Fence 3: the literal lint — which deliberately stays vague

The lint also checks whether any expected answer appears verbatim in the agent's code (a hardcoded lookup table wearing code's clothing). Here's the subtle part: when it finds a violation, the agent is told only VOID: constraint violation — never which string triggered it. Why? Cheat museum #12: a lint that names the offending literal is a membership oracle — the agent could plant guesses, run the scorer, and read the answer key out of the violation messages, one string at a time. The detailed report goes to a file in the vault, addressed to the human. Even the alarm system is on a need-to-know basis.

8.The memorization detector

Suppose the agent hardcodes answers cleverly — obfuscated, encoded, split across branches — and slips past the literal lint. There's a second, deeper detector that catches it from the outside, using pure physics of generalization. It's called the probe, and the idea is beautiful:

Take each practice input and perturb it mechanically, in ways where the right answer is derivable. For the slugify task: uppercase the whole input (the slug shouldn't change, since slugifying lowercases anyway), or append " zz" (the slug should gain -zz).

A program that genuinely implements the algorithm sails through perturbations. A lookup table faceplants, because "CAFÉ AU LAIT!" isn't a key it memorized. The probe gap — dev score minus probe score — is therefore a memorization gauge that requires no access to the agent's code at all. The demo run finished with a probe gap of 0.000: honest work, externally verified.

9.The demo run: 19 cycles to victory

The proving ground was deliberately humble: write slugify.py, a 30-line program that turns "Café au lait!" into cafe-au-lait. (Lowercase, swap accented letters per a given table, collapse junk runs to hyphens, trim.) The task is trivial on purpose — the run was a test of the harness, not the model. Here's the actual score history:

Read that chart honestly and it tells you something most AI demos hide: the curve isn't gradual improvement. It's a wall of zeros while seven harness bugs got found and fixed, then an instant 1.000 once the right model ran with the right context window. Cycle 19: dev 1.000, probe gap 0.000, one audited holdout call → holdout 1.000 ≥ 0.95 → the loop stopped itself. Total cost in tokens: zero dollars. Everything ran on the laptop.

10.Seven failures, seven fixes (the honest part)

The repo's LESSONS.md is a build log of everything that broke. The discipline it documents is the most Feynman thing in the project — "the first principle is that you must not fool yourself." Every fix went into the harness, never into hand-holding the agent mid-run. If the agent cheats, that's a bug in the loss function, not in the agent. Condensed: