Anthropic released Claude Sonnet 5 yesterday. The benchmarks are better, so pointing an existing integration at it looks like an upgrade.

The model is part of the product's behaviour. Suppose it classifies incoming support mail. Sonnet 5 may perform better overall while sending the polite cancellation requests to the questions queue. The prompt stayed the same and the integration still works.

Release benchmarks cannot include the odd cases that matter in your domain. You need examples from production with an expected outcome, especially where the current model already surprised you. Run those against the new model before changing traffic.

We already know this pattern from runtime upgrades. Add the new version to the CI matrix and run the same suite against both. For a model migration, that suite is a fixed set of real inputs with expected outcomes. CI can show where the candidate changed a decision.

Then deploy it in shadow. The current model handles each request while the candidate receives a copy and its result goes into the books without affecting the customer. Over time you learn where they disagree on actual production traffic.

Once those differences are understood, switch. Keep the previous model available until the new behaviour has proved itself outside the shadow deployment.