DSPy: write the program, let an optimizer write the prompts

DSPy replaces hand-tuned prompt strings with small Python programs. You declare what each step takes and returns, compose steps into a module, write a function that scores an output, and run an optimizer. The optimizer runs your program on examples, keeps what scores well, and writes the instructions and few-shot examples for you. When a new model ships, you rerun the optimizer instead of rewriting prompts by hand.

DSPy is an open-source Python framework from the Stanford NLP group, MIT licensed. Version 3.4.1 shipped on 12 October 2026 and supports Python 3.10 to 3.15. It calls models through LiteLLM, so dspy.LM("openai/..."), dspy.LM("anthropic/...") or a local OpenAI-compatible server all work the same way.

Four ideas carry the whole framework:

• Signatures declare a step's inputs and outputs with types: ticket: str in, team: Literal["billing", "technical", "account", "other"] and urgent: bool out. The docstring is the starting instruction. You never write the prompt template.
• Modules decide how a signature is called. dspy.Predict asks directly, dspy.ChainOfThought adds a reasoning field first, dspy.ReAct runs a tool loop, and ProgramOfThought, CodeAct, MultiChainComparison, BestOfN, Refine and Parallel cover other patterns. You compose them in a dspy.Module with a forward() method like any Python class.
• Metrics are plain functions that take the labelled example and the prediction and return a bool, a float, or dspy.Prediction(score=..., feedback=...).
• Optimizers (DSPy also calls them teleprompters) tune one or more of three things: the instructions, the few-shot demos, or the model weights.

In the stack, DSPy is the optimization layer. It sits above the gateway and the model, next to your eval tools: promptfoo, Braintrust or Inspect AI measure a prompt; DSPy searches for a better one against the same kind of metric.

DSPy earns its keep when three things are true: you can score an output automatically, you have labelled examples (a few dozen at least, a few hundred is better), and the prompt drives cost or quality.

• Moving to a smaller or cheaper model. The usual story: optimize a pipeline on a small model with a strong model as the instruction writer, and match or beat the hand-prompted large model. DSPy's own GEPA tutorial reports a nano-sized model climbing from 78% to 90% on its task metric, above the large model's 82% baseline.
• Swapping models. A new model release becomes a rerun of the optimizer and an eval, not a week of prompt edits.
• Multi-step pipelines. Retrieval, classification and generation steps tuned together, where fixing one prompt by hand breaks another.
• Classifiers and extractors with clear labels. Exact-match metrics make optimization cheap and reliable.

Skip it, or wait, when:

• You cannot write a metric. No metric, no optimization. Write the eval first with promptfoo or Braintrust.
• You have a handful of examples. Optimizers overfit tiny sets, especially demo-based ones.
• The current prompt already passes. If a hand-written prompt on your current model meets the bar at an acceptable cost, compiling costs money for no gain.
• You need a fixed, auditable prompt for compliance review. You can still review the compiled instructions (they are saved as JSON), but they change every time you recompile.

The nearest alternatives are hand-iteration in a playground with an eval suite (fine for one prompt), vendor prompt-improver features in hosted tools such as Braintrust's Loop, and fine-tuning a model directly.

You build a program, measure it, compile it, and measure again on data the optimizer never saw:

1. Configure the model. dspy.configure(lm=dspy.LM("openai/gpt-6-luna"), track_usage=True) sets the default. dspy.context(lm=...) overrides it for a block. Calls are cached on disk and in memory by default.
2. Split the data. A training set feeds the optimizer, a validation set picks the winning candidate, and a test set reports the result. GEPA's guide says to keep most data in training and make validation the smallest sample that still represents real traffic.
3. Evaluate the baseline. dspy.Evaluate(devset=test, metric=metric, num_threads=8) runs the program in parallel and returns an EvaluationResult whose .score is a percentage and whose .results holds every (example, prediction, score) triple. A metric that raises scores failure_score (0 by default) instead of stopping the run.
4. Compile. optimizer.compile(program, trainset=train, valset=val) returns a new, optimized copy and leaves the original untouched. Compiled sub-modules are marked so a second optimizer on an outer program leaves them alone.
5. Save and serve. optimized.save("triage.json") writes the instructions, demos and model settings as reviewable JSON. Load it into a freshly built program in production; compile once, serve many times.

The optimizers available in 3.4, grouped by what they tune, from DSPy's own selection guide:

GEPA is the only optimizer that reads the feedback text from your metric. MIPROv2 and COPRO treat the metric as a single number. The docs' rule of thumb: demo tuning tends to overfit small sets, instruction tuning tends to generalize, and fine-tuning is the last lever, not the first.