Given at Airflow Summit 2026, Austin, TX.
Abstract
Building an AI capability in Airflow is the easy part. The hard part is what comes next. You want to swap a model, refactor a prompt, cut token costs, or try a local model instead of paying for cloud. How do you know it still works as expected? Without a fast feedback loop, every change is a gamble. This talk shows practical patterns for building that feedback loop, with real examples using agent skills, MCPs, and local and cloud models. It covers the challenges too: sandboxing, observability, non-determinism, and keeping checks simple enough that people actually use them.
Takeaways
- Build executable checks for your loops, so you can delegate strongly.
- Find the right box: a task, a Dag, the whole Airflow image. Or bigger.
- You may not need the inference at all: AI all around your system, and workflows with none in them.
Evals are not security. If something must never happen, prevent it rather than evaluating it.
Video, Slides and Transcript
- Video on YouTube
- It Works! Now What?
- Transcript (WebVTT), an edited transcript timed to the video
Related
- Find The Loop
- Resumable Task Execution for Long Running Tasks such as Spark, Amogh Desai’s Airflow Summit 2026 talk on the feature I tested
