A structured output contract is what you get when you ask a model to return JSON against a schema. Most teams treat it like a formatting instruction. It is an interface, the kind you’d design between two services, and it has the same failure modes: silent drift, breaking changes shipped without a version bump, and downstream code that trusted a field which quietly stopped existing.
The structured output contract nobody writes down
Every schema you hand a model makes promises on behalf of your whole pipeline, whether you intended it to or not. The field names, the required-versus-optional split, the enum values you allowed, the nesting depth: each of those is a decision your database, your queue consumers and your retry logic now depend on. Nobody writes this down as a contract because it looks like a prompting detail. It’s the schema that every downstream function, every database write, and every retry loop is built against, and it changes what the model is allowed to say.
Why strict json mode doesn’t solve the real problem
Strict json mode guarantees syntactic validity, not semantic correctness. The model will return parseable JSON. It will not reliably return the right values inside it. A date field typed as a string will get a plausible-looking date that’s wrong by a day because the model guessed the timezone. An enum field will get a value close to one of your allowed options but not exactly it, unless you constrain it hard enough that guessing isn’t possible. Enum constraints are the cheapest reliability win available here: give the model a closed list instead of a free-text field, and an entire category of structured output failure modes disappears before you’ve written a line of validation code. This is the same discipline covered in prompting for reliability, not cleverness: the fix is boring and explicit, not clever.
Schema design is where the real engineering happens
The prompt gets the model’s attention. The schema does the actual constraining, and that’s where most of the design effort should go. A field called notes invites a paragraph. A field called cancellation_reason with five allowed enum values invites one of five answers. Nesting matters too: a flat schema is easier for a model to fill correctly than a deeply nested one, because at each level of nesting there’s another place for the model to misjudge required-versus-optional. If you’re retrieving prior state to feed into a schema-constrained call (a customer record, a prior decision, a running total) treat that retrieval step with the same rigor you’d give the schema itself. Loose thinking about what gets pulled in is a big part of why “memory” is the wrong word for what most agents need: it’s state and retrieval, and both have to be well-defined before they hit the schema.
Schema versioning, because the contract will change
You will change this schema, and the question is whether you planned for it. Add a required field to a live function-calling schema and every call made against the old version starts failing validation the moment you deploy, with no warning. The fix is the one every API designer already knows: version the schema explicitly, make new fields optional with sane defaults until every caller has migrated, and never repurpose an existing field name to mean something new. Treat an llm output schema like you’d treat a public API schema, because functionally that’s what it is: consumers you don’t control are reading from it.
What to validate, and what to just retry
Validate structure and types automatically; don’t try to validate judgment calls with more schema. JSON schema validation at the boundary catches malformed output cheaply and should run on every call, no exceptions. But if the model returns syntactically valid output with a wrong classification or a hallucinated field value, more schema rigidity won’t catch that. Only comparing output against a held-out set of known-correct answers will, which is the same quality discipline laid out in measuring quality, not just speed. Retry on validation failure, with a tighter version of the same prompt. Don’t retry on a value that’s merely surprising; that’s a different problem, and conflating the two is how teams end up with retry loops that mask real accuracy issues instead of fixing them.
The schema is the part of this system that will outlive the model you’re using today. Swap providers, fine-tune, change temperature: the schema is still what every downstream consumer reads. Design it like you intend to keep it.