Evalstate Qwen GEPA Whole Dataset Comparison

Run: gepa-evalstate-qwen-overlay-c4-full-20260616T172947Z. Whole 330 GEPA-best scores are from qwen3-6-35b-a3b-nvfp4-thinking-medium-gepa-best-evalstate330-fixed-template-20260617T041156Z.

What Is Plotted

The whole-330 comparison now uses the repaired full GEPA-best run: 330 rows, 7 failed initial rows rerun successfully, 0 remaining errors. Whole-330 bars use the same GEPA-style mean score as the optimizer formula; the metric table also shows micro precision/recall/F1 and exact match.

Whole 330 GEPA mean
0.7350
v10 0.7307, delta +0.0043
Whole 330 Micro-F1
0.8206
v10 0.8231, delta -0.0025
Exact Match
0.5424
v10 0.5242, delta +0.0182
FP / FN
110 / 116
v10 102 / 119

Split Scores With Whole-Dataset Context

0.500.600.700.800.82Pareto 60 v10 seed: 0.57420.5742Pareto 60 GEPA best: 0.69790.6979Pareto 60Heldout 78 v10 seed: 0.76860.7686Heldout 78 GEPA best: 0.78820.7882Heldout 78Whole 330 v10 seed: 0.73070.7307Whole 330 GEPA best: 0.73500.7350Whole 330GEPA mean plottedevaluation set
v10 seedGEPA best

Whole 330 Metrics

0.000.250.500.751.00v10 seed GEPA mean: 0.73070.731GEPA best GEPA mean: 0.73500.735GEPA meanv10 seed Precision: 0.83440.834GEPA best Precision: 0.82460.825Precisionv10 seed Recall: 0.81200.812GEPA best Recall: 0.81670.817Recallv10 seed Micro-F1: 0.82310.823GEPA best Micro-F1: 0.82060.821Micro-F1v10 seed Exact: 0.52420.524GEPA best Exact: 0.54240.542Exactwhole 330 metric
v10 seed repairedGEPA best repaired

Score Table

SetCandidateScoreScore meaning
Pareto 60v10 seed0.5742GEPA/Pareto score
Pareto 60GEPA best0.6979GEPA/Pareto score
Heldout 78v10 seed0.7686GEPA heldout mean score
Heldout 78GEPA best0.7882GEPA heldout mean score
Whole 330v10 seed0.7307GEPA mean score
Whole 330GEPA best0.7350GEPA mean score

Whole Metric Table

Metricv10 seed repairedGEPA best repairedDelta
GEPA mean0.73070.7350+0.0043
Precision0.83440.8246-0.0098
Recall0.81200.8167+0.0047
Micro-F10.82310.8206-0.0025
Exact0.52420.5424+0.0182
False positives102110+8.0000
False negatives119116-3.0000