GEPA Search Result
The search gain on Pareto is much larger than the heldout gain, so treat this as useful prompt evidence, not a final production proof. The heldout still improves, mainly by increasing recall and exact matches.
Scores Across Proposals
This is the GEPA search trajectory for the latest run. The blue line is the full Pareto score for accepted proposals, and the green line is the best score seen so far. Best candidate: proposal 27, score 0.6979.
Heldout Comparison: GEPA Best vs v10 Seed
| Metric | GEPA best | v10 seed | Delta |
|---|---|---|---|
| Mean score | 0.7882 | 0.7686 | +0.0196 |
| Precision | 0.8731 | 0.8889 | -0.0158 |
| Recall | 0.8125 | 0.7778 | +0.0347 |
| Micro-F1 | 0.8417 | 0.8296 | +0.0121 |
| Exact matches | 50 | 46 | +4 |
| False positives | 17 | 14 | +3 |
| False negatives | 27 | 32 | -5 |
| Over-label total | 6 | 4 | +2 |
| Mean predicted labels | 1.7179 | 1.6154 | +0.1026 |
| Rows | 78 | 78 | same |
This does not look like a random-more-labels shortcut: mean predicted labels rose by about 0.10 per row, exact matches improved by 4, and precision stayed high. The tradeoff is real, though: GEPA bought recall with 3 extra false positives and 2 extra over-labels.
Whole 330 Comparison
| Metric | GEPA best repaired | v10 seed repaired | Delta |
|---|---|---|---|
| GEPA mean score | 0.7350 | 0.7307 | +0.0043 |
| Micro-F1 | 0.8206 | 0.8231 | -0.0025 |
| Precision | 0.8246 | 0.8344 | -0.0098 |
| Recall | 0.8167 | 0.8120 | +0.0047 |
| Exact match | 0.5424 | 0.5242 | +0.0182 |
| False positives | 110 | 102 | +8 |
| False negatives | 116 | 119 | -3 |
On the whole 330 rows, GEPA-best narrowly improves the GEPA-style mean score and exact match, but v10 keeps a slightly better micro-F1 because GEPA-best trades 3 fewer false negatives for 8 more false positives.
Run Config
| Model | nvidia/Qwen3.6-35B-A3B-NVFP4 |
|---|---|
| Runtime | localpager-agent via scripts/localpager-classifier |
| Endpoint | http://127.0.0.1:8000/v1 |
| Concurrency | 4 |
| Thinking | medium |
| Max output tokens | 8192 |
| Splits | feedback300 train, pareto60 Pareto, bench78 heldout |
| Prompt setup | v10 overlay-only scaffold seeded from localpager-openclaw-routing-v10-overlay-seed.md |
| Taxonomy | openclaw-routing-topics.v2.json |
| Budget | max_metric_calls=1440, max_candidate_proposals=32, reflection_minibatch_size=4 |
| Reflection LM | CodexReflectionLM |
Artifacts
Whole dataset comparison Proposal graphs Score graph Prompt diff picker Candidate tree Best rendered prompt Best routing policy GEPA summary JSON Heldout best JSON Heldout seed JSON
- No manual prompt edits were made to the GEPA best candidate.
- The run only mutated the overlay, not the schema, taxonomy, or label enum.
- Searches found no hardcoded row IDs, GitHub URLs, gold labels, or split names in the selected prompt/policy.
- The first heldout-best slice had local subprocess failures; the failed rows were rerun cleanly after adding diagnostics, then combined with the successful rows.