search

adiii717 / docker-python-cronjob

Run python script as a cron job using Docker
93 stars 45 forks source link

No output using this example #1

Open SagarKevin opened 3 years ago

SagarKevin commented 3 years ago

Hi, Thank you very much for this example. I used the exact same files and code, but there was no output generated. Would you happen to know why that is the case? Thanks1

adiii717 commented 3 years ago

@SagarKevin seems like you are missing something at your end? as its not expected behavior. Also, make sure you wait for 1 minute as the job executes every minute.

image

SagarKevin commented 3 years ago

@Adiii717 thanks for your reply. Below is the log from the console. I followed all the steps mentioned and used the exact code and naming convention you used. image

Files: image image image

I let it it run for a few minutes, but still no output. Not sure what the issue is here. is there a way I can troubleshoot this? Thanks

gekakud commented 3 years ago

@Adiii717 same here, as @SagarKevin reported.

adiii717 commented 3 years ago

@gekakud which os you are using and docker version please

adiii717 commented 3 years ago

@SagarKevin one way to check container logs

docker ps  # copy container name|id
docker logs -f container_name
barcalalor commented 3 years ago

Hello guys,

I find the fix to the problem of this repo,

You have to replace the crontab by:

run python script every minutes

And thanks to Adii for the repo

Coding-Crashkurse commented 2 years ago

@barcalalor : This does not fix the issue for me :-(. Any further suggestions?

thomas-romuald commented 2 years ago

Hi, I cloned this repo to try this example. looks like the cron works but the pyhton command does not work.

Build:

$ docker build -t python-cron .
Sending build context to Docker daemon  614.4kB
Step 1/8 : FROM python:3.7
 ---> 3ce71eee90f7
Step 2/8 : RUN apt-get update && apt-get -y install cron vim
 ---> Running in 12341fe25b19
Get:1 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Get:2 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Get:4 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [97.5 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 Packages [8180 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2592 B]
Fetched 8479 kB in 3s (2507 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  exim4-base exim4-config exim4-daemon-light gsasl-common guile-2.2-libs
  libgc1 libgnutls-dane0 libgpm2 libgsasl7 libidn11 libmailutils7 libntlm0
  libpython3.9 libunbound8 mailutils mailutils-common psmisc vim-common
  vim-runtime xxd
Suggested packages:
  anacron logrotate checksecurity exim4-doc-html | exim4-doc-info eximon4
  spf-tools-perl swaks dns-root-data gpm mailutils-mh mailutils-doc ctags
  vim-doc vim-scripts
The following NEW packages will be installed:
  cron exim4-base exim4-config exim4-daemon-light gsasl-common guile-2.2-libs
  libgc1 libgnutls-dane0 libgpm2 libgsasl7 libidn11 libmailutils7 libntlm0
  libpython3.9 libunbound8 mailutils mailutils-common psmisc vim vim-common
  vim-runtime xxd
0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded.
Need to get 21.2 MB of archives.
After this operation, 102 MB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 cron amd64 3.0pl1-137 [99.6 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 xxd amd64 2:8.2.2434-3 [192 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 vim-common all 2:8.2.2434-3 [226 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 exim4-config all 4.94.2-7 [335 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 exim4-base amd64 4.94.2-7 [1175 kB]
Get:6 http://deb.debian.org/debian bullseye/main amd64 libunbound8 amd64 1.13.1-1 [504 kB]
Get:7 http://deb.debian.org/debian bullseye/main amd64 libgnutls-dane0 amd64 3.7.1-5 [394 kB]
Get:8 http://deb.debian.org/debian bullseye/main amd64 libidn11 amd64 1.33-3 [116 kB]
Get:9 http://deb.debian.org/debian bullseye/main amd64 exim4-daemon-light amd64 4.94.2-7 [658 kB]
Get:10 http://deb.debian.org/debian bullseye/main amd64 gsasl-common all 1.10.0-4 [175 kB]
Get:11 http://deb.debian.org/debian bullseye/main amd64 libgc1 amd64 1:8.0.4-3 [239 kB]
Get:12 http://deb.debian.org/debian bullseye/main amd64 guile-2.2-libs amd64 2.2.7+1-6 [4980 kB]
Get:13 http://deb.debian.org/debian bullseye/main amd64 libgpm2 amd64 1.20.7-8 [35.6 kB]
Get:14 http://deb.debian.org/debian bullseye/main amd64 libntlm0 amd64 1.6-3 [84.7 kB]
Get:15 http://deb.debian.org/debian bullseye/main amd64 libgsasl7 amd64 1.10.0-4 [195 kB]
Get:16 http://deb.debian.org/debian bullseye/main amd64 mailutils-common all 1:3.10-3 [728 kB]
Get:17 http://deb.debian.org/debian bullseye/main amd64 libpython3.9 amd64 3.9.2-1 [1691 kB]
Get:18 http://deb.debian.org/debian bullseye/main amd64 libmailutils7 amd64 1:3.10-3+b1 [893 kB]
Get:19 http://deb.debian.org/debian bullseye/main amd64 mailutils amd64 1:3.10-3+b1 [576 kB]
Get:20 http://deb.debian.org/debian bullseye/main amd64 psmisc amd64 23.4-2 [198 kB]
Get:21 http://deb.debian.org/debian bullseye/main amd64 vim-runtime all 2:8.2.2434-3 [6226 kB]
Get:22 http://deb.debian.org/debian bullseye/main amd64 vim amd64 2:8.2.2434-3 [1494 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 21.2 MB in 1s (14.9 MB/s)
Selecting previously unselected package cron.
(Reading database ... 23378 files and directories currently installed.)
Preparing to unpack .../00-cron_3.0pl1-137_amd64.deb ...
Unpacking cron (3.0pl1-137) ...
Selecting previously unselected package xxd.
Preparing to unpack .../01-xxd_2%3a8.2.2434-3_amd64.deb ...
Unpacking xxd (2:8.2.2434-3) ...
Selecting previously unselected package vim-common.
Preparing to unpack .../02-vim-common_2%3a8.2.2434-3_all.deb ...
Unpacking vim-common (2:8.2.2434-3) ...
Selecting previously unselected package exim4-config.
Preparing to unpack .../03-exim4-config_4.94.2-7_all.deb ...
Unpacking exim4-config (4.94.2-7) ...
Selecting previously unselected package exim4-base.
Preparing to unpack .../04-exim4-base_4.94.2-7_amd64.deb ...
Unpacking exim4-base (4.94.2-7) ...
Selecting previously unselected package libunbound8:amd64.
Preparing to unpack .../05-libunbound8_1.13.1-1_amd64.deb ...
Unpacking libunbound8:amd64 (1.13.1-1) ...
Selecting previously unselected package libgnutls-dane0:amd64.
Preparing to unpack .../06-libgnutls-dane0_3.7.1-5_amd64.deb ...
Unpacking libgnutls-dane0:amd64 (3.7.1-5) ...
Selecting previously unselected package libidn11:amd64.
Preparing to unpack .../07-libidn11_1.33-3_amd64.deb ...
Unpacking libidn11:amd64 (1.33-3) ...
Selecting previously unselected package exim4-daemon-light.
Preparing to unpack .../08-exim4-daemon-light_4.94.2-7_amd64.deb ...
Unpacking exim4-daemon-light (4.94.2-7) ...
Selecting previously unselected package gsasl-common.
Preparing to unpack .../09-gsasl-common_1.10.0-4_all.deb ...
Unpacking gsasl-common (1.10.0-4) ...
Selecting previously unselected package libgc1:amd64.
Preparing to unpack .../10-libgc1_1%3a8.0.4-3_amd64.deb ...
Unpacking libgc1:amd64 (1:8.0.4-3) ...
Selecting previously unselected package guile-2.2-libs:amd64.
Preparing to unpack .../11-guile-2.2-libs_2.2.7+1-6_amd64.deb ...
Unpacking guile-2.2-libs:amd64 (2.2.7+1-6) ...
Selecting previously unselected package libgpm2:amd64.
Preparing to unpack .../12-libgpm2_1.20.7-8_amd64.deb ...
Unpacking libgpm2:amd64 (1.20.7-8) ...
Selecting previously unselected package libntlm0:amd64.
Preparing to unpack .../13-libntlm0_1.6-3_amd64.deb ...
Unpacking libntlm0:amd64 (1.6-3) ...
Selecting previously unselected package libgsasl7:amd64.
Preparing to unpack .../14-libgsasl7_1.10.0-4_amd64.deb ...
Unpacking libgsasl7:amd64 (1.10.0-4) ...
Selecting previously unselected package mailutils-common.
Preparing to unpack .../15-mailutils-common_1%3a3.10-3_all.deb ...
Unpacking mailutils-common (1:3.10-3) ...
Selecting previously unselected package libpython3.9:amd64.
Preparing to unpack .../16-libpython3.9_3.9.2-1_amd64.deb ...
Unpacking libpython3.9:amd64 (3.9.2-1) ...
Selecting previously unselected package libmailutils7:amd64.
Preparing to unpack .../17-libmailutils7_1%3a3.10-3+b1_amd64.deb ...
Unpacking libmailutils7:amd64 (1:3.10-3+b1) ...
Selecting previously unselected package mailutils.
Preparing to unpack .../18-mailutils_1%3a3.10-3+b1_amd64.deb ...
Unpacking mailutils (1:3.10-3+b1) ...
Selecting previously unselected package psmisc.
Preparing to unpack .../19-psmisc_23.4-2_amd64.deb ...
Unpacking psmisc (23.4-2) ...
Selecting previously unselected package vim-runtime.
Preparing to unpack .../20-vim-runtime_2%3a8.2.2434-3_all.deb ...
Adding 'diversion of /usr/share/vim/vim82/doc/help.txt to /usr/share/vim/vim82/doc/help.txt.vim-tiny by vim-runtime'
Adding 'diversion of /usr/share/vim/vim82/doc/tags to /usr/share/vim/vim82/doc/tags.vim-tiny by vim-runtime'
Unpacking vim-runtime (2:8.2.2434-3) ...
Selecting previously unselected package vim.
Preparing to unpack .../21-vim_2%3a8.2.2434-3_amd64.deb ...
Unpacking vim (2:8.2.2434-3) ...
Setting up libgpm2:amd64 (1.20.7-8) ...
Setting up psmisc (23.4-2) ...
Setting up cron (3.0pl1-137) ...
Adding group `crontab' (GID 102) ...
Done.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up xxd (2:8.2.2434-3) ...
Setting up libunbound8:amd64 (1.13.1-1) ...
Setting up libpython3.9:amd64 (3.9.2-1) ...
Setting up vim-common (2:8.2.2434-3) ...
Setting up libntlm0:amd64 (1.6-3) ...
Setting up libidn11:amd64 (1.33-3) ...
Setting up mailutils-common (1:3.10-3) ...
Setting up libgc1:amd64 (1:8.0.4-3) ...
Setting up exim4-config (4.94.2-7) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Adding system-user for exim (v4)
Setting up gsasl-common (1.10.0-4) ...
Setting up vim-runtime (2:8.2.2434-3) ...
Setting up guile-2.2-libs:amd64 (2.2.7+1-6) ...
Setting up libgnutls-dane0:amd64 (3.7.1-5) ...
Setting up exim4-base (4.94.2-7) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
exim: DB upgrade, deleting hints-db
Setting up vim (2:8.2.2434-3) ...
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vim (vim) in auto mode
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vimdiff (vimdiff) in auto mode
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rvim (rvim) in auto mode
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/rview (rview) in auto mode
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/vi (vi) in auto mode
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/view (view) in auto mode
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/ex (ex) in auto mode
update-alternatives: using /usr/bin/vim.basic to provide /usr/bin/editor (editor) in auto mode
Setting up libgsasl7:amd64 (1.10.0-4) ...
Setting up exim4-daemon-light (4.94.2-7) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up libmailutils7:amd64 (1:3.10-3+b1) ...
Setting up mailutils (1:3.10-3+b1) ...
update-alternatives: using /usr/bin/frm.mailutils to provide /usr/bin/frm (frm) in auto mode
update-alternatives: using /usr/bin/from.mailutils to provide /usr/bin/from (from) in auto mode
update-alternatives: using /usr/bin/messages.mailutils to provide /usr/bin/messages (messages) in auto mode
update-alternatives: using /usr/bin/movemail.mailutils to provide /usr/bin/movemail (movemail) in auto mode
update-alternatives: using /usr/bin/readmsg.mailutils to provide /usr/bin/readmsg (readmsg) in auto mode
update-alternatives: using /usr/bin/dotlock.mailutils to provide /usr/bin/dotlock (dotlock) in auto mode
update-alternatives: using /usr/bin/mail.mailutils to provide /usr/bin/mailx (mailx) in auto mode
Processing triggers for libc-bin (2.31-13+deb11u2) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Removing intermediate container 12341fe25b19
 ---> ee09c8f4f850
Step 3/8 : WORKDIR /app
Removing intermediate container fff9c16d000a
 ---> 94f0418a51ea
Step 4/8 : COPY crontab /etc/cron.d/crontab
 ---> bf1b3209c025
Step 5/8 : COPY hello.py /app/hello.py
 ---> 913558f46fe6
Step 6/8 : RUN chmod 0644 /etc/cron.d/crontab
 ---> Running in d0e7286492a8
Removing intermediate container d0e7286492a8
 ---> edb25b494b60
Step 7/8 : RUN /usr/bin/crontab /etc/cron.d/crontab
 ---> Running in c0df0f6e3597
Removing intermediate container c0df0f6e3597
 ---> 615016377ae3
Step 8/8 : CMD ["cron", "-f"]
 ---> Running in ef256fc93969
Removing intermediate container ef256fc93969
 ---> 75cf56adff83
Successfully built 75cf56adff83
Successfully tagged python-cron:latest

Run:

$ docker run -it --rm python-cron
/bin/sh: 1: python: not found
/bin/sh: 1: python: not found
/bin/sh: 1: python: not found
/bin/sh: 1: python: not found
/bin/sh: 1: python: not found

Docker version:

$ docker -v
Docker version 18.03.1-ce, build 9ee9f40
leranp commented 2 years ago

Same here

gist901 commented 2 years ago

For the issue with python: not found you need to update the crontab file by adding a 3 to the end of python and it should work after that. So your new crontab file command should look like this:

#run python script every minutes
* * * * * python3 /app/hello.py > /proc/1/fd/1 2>/proc/1/fd/2
#Empty line for validation
mkrijtenberg commented 1 year ago

I've got it working by modifying the crontab:

* * * * * python3 /app/hello.py > /var/log/cron.log 2>/var/log/cron.log

and add this to the Dockerfile (before the CMD):

RUN ln -sf /proc/1/fd/1 /var/log/cron.log

UrsDeSwardt commented 1 year ago

I spent quite a while trying to get this to work. It turns out I had to change my spacing setting from CRLF to LF in VS Code for the crontab file.

image

git-shogg commented 1 year ago

Ah right, the "LF" method didn't work for me 😢. I figure it must be something to do with the cron job as when I copy paste my python command from my crontab file to the docker terminal that seems to execute fine and even output to the right logs. Anyone else manage to get this one working?

git-shogg commented 1 year ago

My issue was resolved and the cron began working by switching from python:3.8-slim to python:3.10! Slim images must have a configuration that messes with cron, could be the same for other image types... For future reference, my dockerfile and cron looked as follows:

FROM python:3.10
RUN apt-get update && apt-get -y install cron vim
WORKDIR /services/app

COPY ../finsec ./finsec
COPY ../../requirements.txt requirements.txt
COPY /services/app .
COPY /services/app/crontab /etc/cron.d/crontab

RUN pip install -r requirements.txt

RUN chmod 0644 /etc/cron.d/crontab

RUN crontab /etc/cron.d/crontab

CMD ["cron", "-f"]
* * * * * python3 /services/app/main.py > /proc/1/fd/1
elburro1887 commented 1 year ago

@git-shogg Thanks for the tip pertaining to the slim image. I switched to the normal image, but then I encountered the following fix and I found out it doesn't matter if I use the slim image or not. I'm using python:3.9-slim-bullseye now.

So the actual problem in my case was that python3 was not picking up dependencies installed with pip (apparently using pip installs them for the local user's python only (-> see https://stackoverflow.com/a/59572201/11764914 for example).

So in the crontab I now use the user's local python and it works.

* * * * * /usr/local/bin/python /task.py > /dev/stdout 2>&1

git-shogg commented 1 year ago

Thanks @elburro1887, I'll check out your approach!