LLM Seeded Genetic Search

A field guide to the technique: the idea, a real solved task end to end, and the raw material behind the paper.

Companion page to our arXiv paper. This page carries the detail that doesn't belong in a formal paper, full LLM conversations, real solved programs, and additional charts. Numbers here get updated as we run more of the task set, so treat this as a living page rather than a snapshot.

The core idea

ARC-AGI-2 tasks give you a handful of input→output grid examples and ask you to figure out the rule, then apply it to a new grid. We build solutions to these puzzles by combining two things that are each, on their own, not very good at this job.

A large language model, on its own, is fine but slow. It can reason about a task's transformation rule in plain language, and it often proposes something plausible on the first try. But every attempt costs a real LLM call, and the number of attempts you can afford is bounded by hardware, how many copies of the model you can run, not by any real ceiling on how many variations might be worth trying.

A genetic algorithm, on its own, basically doesn't work. We tried it: seed a population entirely at random over our 41-operation DSL and let it evolve, no LLM involved anywhere. With chromosomes capped at 20 operations, the raw search space is around 41²⁰ operation sequences, and that's before you even count the five arguments each operation takes. Blind search over that space essentially never lands on an exact answer in a reasonable number of generations.

So we built a bridge between them.

Every so often the LLM's own proposal is already exact, no search needed at all. We don't have a tidy explanation for why that happens as often as it does.

The LLM doesn't write one guess. It's prompted to propose a broad, generous set of candidate programs: the leading hypothesis, plus every plausible variant of it. That set becomes the seed material for a genetic algorithm that runs entirely on the GPU as a single batched tensor operation, with zero further LLM calls, evolving the population toward an exact pixel match.

The Machine

The "Machine" is the shared language both the LLM and the genetic algorithm address directly. It has three parts:

Perception, turning a raw grid into registers and objects, is deterministic and greedy: color-channel splitting plus neighbor connectivity. It's a simple first pass, and we know it misses groupings a person would spot immediately. But it gives every chromosome, every prompt, and every mutation a shared vocabulary to point at, "the dominant color," "object 3's bounding box", instead of hard-coding values that would only make sense for one training pair.

A real task, start to finish

Here's an actual run, unedited except for length, on task 017c7c7b, a 6×3 input mapped to a 9×3 output. This one's a genuine, verified solve: we checked the winning chromosome against the real test grid ourselves, not just the training pairs.

The LLM's step is a fixed four-stage conversation. Below is that conversation, real and mostly unabridged (the full operation catalogue and a few argument lists are elided for length, marked with […] exactly where our source notes elided them).

You are an ARC-AGI puzzle expert acting as the front end of a program-synthesis search. A genetic algorithm takes the candidate programs you write, runs them on the puzzle, and mutates and recombines them over many generations to find one that reproduces every training example.
Your goal is NOT to nail the one correct program. It is to (a) work out the most likely rule, then (b) hand the search a broad, generous set of programs that bracket it: the leading hypothesis, every plausible variant of it (different channels, directions, colors, coordinate registers, gene order), and the back-up reads.
[… full catalogue of all 41 ops with args and examples, the register list, chromosome-syntax rules …]

task 0 input (6x3)
010
101
010
101
010
101
task 0 output (9x3)
020
202
020
202
[…]
task 0 channels
OUT: color 1, 9 cells, in bbox rows 0..5 cols 0..2
col1: color 1, 9 cells…
obj0..obj7: the 8 isolated single-cell objects
task 0 registers
H=6 W=3 Hh=3 Wh=1 | bg=0 dom=1 | n_objects=8 n_colors=1
centroid=(2.7,1) bbox_center=(…)

"The transformation maps a 6x3 input grid of color 1 cells to a 9x3 output grid of color 2 cells. Dimensions: input is 6x3, output is 9x3, height increases by 3 rows, width stays the same. Color 1 becomes color 2. Input rows with 1s: 0,1,2,3,4,5. Output rows with 2s: 0,1,(gap),3,4,(gap),6,7,8…"
(continues reasoning toward: the pattern periodically repeats downward with a gap inserted every 2 rows.)

Which ops from the Op list could carry out that rule? Pick the ones that fit best. For each picked op: quote its catalogue line, then in one phrase say how it maps to the rule… Don't restate the rule itself.

copy <col1> OUT
scale <col1> k0 k0 OUT
reg_add k1 k1 sr0
reg_select k0 k0 k1 sr0
extend <col1> (sr0) k1 k0 2
copy_paste <col1> down k3
panel_bool 1 2 OUT

You're not sure of the exact arguments yet. For each op you picked, list every value worth trying for each argument, be exhaustive: which channel it acts on… direction registers for dr,dc… which register supplies each coordinate argument… Only fall back to a bare constant when the position is genuinely fixed.

copy <col1> OUT
- dst: OUT
- src: col1 (or obj0)
scale <col1> k0 k0 OUT
- fr: k0, k1, k2, k3, k4, k5, H, W, obj0_h, obj0_w
- fc: k0, k1, k2, k3, k4, k5, H, W, obj0_h, obj0_w
reg_add k1 k1 sr0
- a: k0..k9, H, W, obj0_cr, obj0_cc, step_dr, step_dc, span_dr, span_dc
- dst: sr0 (or sr1..sr3)
[… continues for every picked op …]

A chromosome is one complete executable hypothesis for the entire task. Genes run sequentially on the state left by earlier genes… Only OUT is scored after the final gene… Do not describe a solution; emit only executable chromosomes that can produce the complete final OUT. One chromosome must implement one complete rule end to end.

copy_paste col1 down k3
; recolor col1 2
; extend_all k1 k0 2
reg_add H k1 sr0
reg_sub k3 sr0 sr1
copy_paste col1 down sr0
panel_bool 2 2 OUT
copy <col1> OUT
reg_add k3 H sr0
extend <col1> sr0 k1 k0 2
reg_add obj0_h k3 sr0
reg_select obj0_r0 obj0_h k3 sr0
copy_paste col1 down sr0
[… a dozen more independent chromosome hypotheses …]

None of these four chromosomes was an exact match on their own, the best scored 53% mean pixel accuracy against the training pairs. They became the seed pool for both GA populations, which reached an exact match by recombining and mutating around them. (Across different runs of this same task we've seen the exact match land anywhere from generation 22 to 25, ordinary run-to-run noise.) We then checked the winning chromosome against the real held-out test grid, and it's correct: this is one of the 6 verified solves in our 60-task sample.

The 41-operation language

A chromosome is a sequence of 20 genes; each gene is an opcode plus five arguments. 27 "core" operations are always available for mutation to draw on. 14 "gated" operations are narrower and more expensive, and only enter a task's search space if the LLM itself proposed one, since they're too specific for random mutation to stumble into usefully on its own, but cheap to unlock the moment the LLM's reasoning actually calls for one.

What we've found so far

Full results, tables, and confidence intervals live in the paper. Here we want to walk through the two findings that matter most, with the actual charts.

Best (solid) and mean (dashed) population fitness by generation, for three verified solves and three unsolved tasks. All three solved examples here were checked against the real test grid, not just the training pairs. Solved tasks tend to show a staircase pattern, long plateaus, then a sudden jump, though one of the three (0c9aba6e) climbs gradually instead, with the whole population improving together rather than one lucky mutation breaking through.

The clearest evidence we have that the LLM's contribution isn't redundant with the genetic algorithm's: the same three tasks, with LLM seeding (solved exactly and verified correct, every time) versus a matched control with zero LLM-derived material (never exact, even after 200 generations).

One honest miss, worth reporting because it's informative: an early hypothesis, from a small 3-task sample, suggested that how many candidate chromosomes the LLM proposes predicts how fast the genetic algorithm converges. Retested on the larger 60-task sample, using only the verified solves, it didn't hold up. We're leaving that in the paper as a negative result instead of quietly dropping it.

Open questions

• Why does an exact training fit sometimes just not generalize? Three of our nine candidate solves matched every training example perfectly and still got the test grid wrong. We don't yet have a good story for what separates those from the six that generalized.
• Why does the LLM ever produce an exact answer at all? It happens more than chance would suggest, on tasks it was never trained to solve, using a DSL it's only seen in-context. We still don't have an explanation.
• The no-LLM control needs to run at full scale. Right now it's three tasks, matched against ones we already solve with LLM help. The real test is running it across the full set, including tasks we currently can't solve at all, and with the same 41-op vocabulary in both conditions so we're isolating one variable at a time.
• What drives chromosome attrition? On some tasks over 90% of what the LLM proposes fails basic syntax verification before it ever reaches the genetic algorithm. We don't know yet why some tasks are so much worse than others.