reliability

Reliability in Django and Celery: how tasks get lost

Every Django project I have worked on reaches the same moment. A request is too slow, someone moves the slow part into a Celery task, and the page gets fast. Quietly, the function has stopped being a function. It became a message, handed to a broker you do not control, to be run by a worker that might die halfway through. This post is my study guide for everything that can go wrong in that handoff, and what to do about each case.