Open kousu opened 7 years ago
Relatedly, the first cronjob needs quoting. It should be
@daily /usr/bin/docker exec -it souschef_web_1 python src/manage.py generateorders $(date +\%Y-\%m-\%d) --days 30
Also, you need to make sure the user that's run as has permissions to docker, either via sudo or by being in the docker group; a note to this effect in the docs would help.
@kousu Thanks for the heads-up! I never tested this part during my dev. I'll check it out according to your comment and update the docs.
Celery and django-cron are overkill for our purpose. We expect it being executed only once a day and we don't expect to re-program the job later.
There was another bug with these: they say to run docker exec -it
but when run by cron that generates the poorly explained "the input device is not a TTY". The answer there says that any of -i
, -t
or ` should work, but not
-it`. I tried that and now the cronjobs are working on DEV, hooray. I think the root of the problem here is Docker and its poor docs.
The manpage describes
-i, --interactive[=false]
Keep STDIN open even if not attached
-t, --tty[=false]
Allocate a pseudo-TTY
which makes it sound like -it
should behave like running the command inside of screen
: it will think its connected to a terminal (via isatty()
) and have an open stdin even when its not either of those things.
We can simulate the effect of crond
with </dev/null
:
sfl@sous-chef-test:~/sous-chef$ docker exec -it souschef_web_1 ls </dev/null
the input device is not a TTY
As the SO answer says, using any combination except for -it
together works:
sfl@sous-chef-test:~/sous-chef$ docker exec -i souschef_web_1 ls </dev/null
CONTRIBUTING.md
Dockerfile
INSTALL.md
LICENSE
README.adoc
deploy_latest.sh
docker-compose.override.yml
docker-compose.prod.yml
docker-compose.travis.yml
docker-compose.yml
docs
mock_addresses.csv
mock_clients.csv
mock_meals.csv
mock_orders.csv
mock_relationships.csv
requirements.txt
src
tools
sfl@sous-chef-test:~/sous-chef$ docker exec -t souschef_web_1 ls </dev/null
CONTRIBUTING.md docker-compose.prod.yml mock_orders.csv
Dockerfile docker-compose.travis.yml mock_relationships.csv
INSTALL.md docker-compose.yml requirements.txt
LICENSE docs src
README.adoc mock_addresses.csv tools
deploy_latest.sh mock_clients.csv
docker-compose.override.yml mock_meals.csv
sfl@sous-chef-test:~/sous-chef$ docker exec souschef_web_1 ls </dev/null
CONTRIBUTING.md
Dockerfile
INSTALL.md
LICENSE
README.adoc
deploy_latest.sh
docker-compose.override.yml
docker-compose.prod.yml
docker-compose.travis.yml
docker-compose.yml
docs
mock_addresses.csv
mock_clients.csv
mock_meals.csv
mock_orders.csv
mock_relationships.csv
requirements.txt
src
tools
I don't understand what docker is doing here. What you wrote should work, to me. It smells like a docker bug, but in the meantime can you update the cronjob.adoc
to remove the -it
--- just using ` works for me, but maybe
-i` is a little , and leave a comment for yourselves somewhere that docker + cron glitch out when combined like this?
Thank you for all your diligent work.
^ Ran into the above issue again today with sous-chef. An update would be greatly appreciated!
Notice that #822 means we need a third cronjob now:
python3 manage.py setordersdelivered $(date --date="yesterday" +"%Y-%m-%d")
and remember to double-quote those percent-signs when you put them into the crontab! because cron adds even more obscure quoting rules on top of shell!
For reference, what I actually have installed right now in the crontab for the user docker is run as:
# the 2>&1 | tees are for debugging: you can always inspect the last run with them
# (aka |& tee ?)
@daily /usr/bin/docker exec souschef_web_1 python src/manage.py generateorders $(date +\%Y-\%m-\%d) --days 100 2>&1 | tee /tmp/generateorders
@daily /usr/bin/docker exec souschef_web_1 python src/manage.py processscheduledstatuschange 2>&1 | tee /tmp/processscheduledstatuschange
@daily /usr/bin/docker exec souschef_web_1 python src/manage.py setordersdelivered $(date --date="yesterday" +"\%Y-\%m-\%d") 2>&1 | tee /tmp/setordersdelivered
Another issue: the example cronjobs suggest
docker exec -it souschef_db_1 mysqldump -uroot -p123456 feast > souschef_data.sql
this is insecure, especially when used with cron, because cron in its default config emails the command line if there's any output to the user running the job (aka their ~/.forward
file, usually, if they're a daemon user and you've set that up properly).
I think this bug needs an edit to the docker config; db
needs to gain a ~/.my.cnf
file, or maybe docker-compose.yml
needs to set MYSQL_PWD
.
It wasn't obvious to me in installing Sous-Chef that there were cronjobs needed. INSTALL.md doesn't say anything about them. And though eventually cronjob.adoc was pointed out to me, I just found out today that it is missing
processscheduledstatuschange
.Apparently the correct cronjob setup is
I would like to request cronjob.adoc be deleted and merged into INSTALL.md.
Also it's probably better to write those jobs as
to handle the case of multiple parallel Sous-Chefs on a single host (which is, afterall, why they're containerized isn't it?)
The install process would be more reliable if there wasn't anything manual at all like this. Both http://www.celeryproject.org/ and https://github.com/Tivix/django-cron can be used to do cron-like things but driven by Django itself; maybe they are useful?