Django Cron with Docker and Gunicorn

[hemangsk]

Hello Martin 😃 I read your awesome blog on https://www.amon.cx/blog/deploying-web-apps-docker/ and I've been searching for a write-up like this on docker + Django + cron for like a month. Thank You!

I was wondering if you can guide me to solve a little Docker-Django issue if it's not too much trouble.

We are having a very straightforward use case, There is Gunicorn powered Django App in Docker, and two cronjobs which fetch data from third party API and store it in Django App's DB. But the issue is I'm not able to find an approach to do this.

Our Django App works like a charm with Gunicorn, inside Docker. But the cronjobs are not working. They are registered for sure because I checked with crontab -l and it showed the cronjobs, but they are not running.

Currently, we're using the Django-Crontab plugin, to register the cronjobs. Here's our Dockerfile https://gitlab.com/coala/landing/blob/master/Dockerfile So this line https://gitlab.com/coala/landing/blob/master/Dockerfile#L15 gets executed successfully, registers the cronjob. Here' the cron script https://gitlab.com/coala/landing/blob/master/backend/bears/cron.py

It'd be really really awesome if you can point me towards a way of achieving this use case. 😃 😃 Thanks a ton!

[martinrusev]

@hemangsk Cron in Docker is not a great idea in general. Docker works well with "stateless" apps, which means you can have unspecified number of containers running and this does not change the state of your app. If you have cronjobs, usually these have to be executed only on the "master" / leader container.

I personally solve the issue by having an internal API and consuming / sending tasks to a separate worker. An example would be ElasticBeanstalk with this http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html

If you absolutely have to run the cronjobs, supevisord is what you are looking for.

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false              ; (start in foreground if true;default false)

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[program:backend]
command=gunicorn -c gunicorn.conf wsgi
directory=/app
autostart=true
autorestart=true
stdout_logfile=/var/log/backend.log
redirect_stderr=true 
stopsignal = QUIT

[program:cron]
command = cron -f -L 15
autostart=true
autorestart=true

And then you have a docker entrypoint that runs supervisor:

#!/bin/bash
set -e
set -u

supervisord -n -c /etc/supervisord.conf

[hemangsk]

Thank you so much! I'll get right on it. This is so helpful for me, I will get started with supervisord straightaway.