Zero-downtime deploys are a solved problem for ordinary services. AI-powered ones add a twist: models are large, slow to load, and expensive to keep warm. The principles hold, but the details need care.
Separate the model from the deploy
Do not bake giant model weights into every release artifact. Load them from a shared, versioned store so a code deploy does not mean re-downloading gigabytes. Your application rolls forward quickly; the model lifecycle is managed on its own cadence.
Warm before you cut over
A new instance that has not loaded its model will serve slow first requests. Health checks should pass only once the model is loaded and a warm-up inference has run. Combine that with graceful draining — reload, do not hard-restart — and the cutover stays invisible.
Roll back in seconds
Keep the previous release ready to take traffic instantly. A bad deploy should be a one-command revert, not a 2am incident. This is exactly the pipeline that let me deploy a patient-facing API continuously without anyone noticing.
Continuous delivery and heavy models are not in conflict — you just respect the model’s weight.