Docker's cron job not working

nilsnolde commented 4 years ago

I was just working on a small PR to expose more env vars to control the databases to be backed up and a few other things.

While testing, I realized, that even the current master doesn't run the cron job configured in the Docker container, at least for me. I feel a bit stupid right now. I changed the backups-cron to run the job every minute, but it wouldn't ever run. I tried execing into the container, starting cron (even though top showed me it was running). I tried adding a simple echo in backups-cron to be run every minute. Nothing ever worked.

However, I could run /backups.sh successfully in the container. Also in a way that cron will approx run it: /bin/sh -c "(export PATH=/usr/bin:/bin; /backups.sh 2>&1)".

Really, the only thing I tried which got it working, was changing crontab from within the container and adding the minutely executed backups.sh right in there. Nothing else seemed to trigger the cron job.

Anyone has experienced that before? I already dumped 3 hours and have no idea what's wrong with the container's cron setup, actually looks all fine to me.

NyakudyaA commented 4 years ago

hi @nilsnolde Can you try to override the entrypoint to something like

entrypoint:
    - /bin/bash start.sh

nilsnolde commented 4 years ago

Thanks for the quick answer @NyakudyaA . Unfortunately that didn't help:

  entrypoint:
    - /bin/bash
    - /start.sh

and the following backups-cron

# For testing - run every minute
* * * * * root /backups.sh 2>&1

# Run the backups at 11pm each night
#0 23 * * * root /backups.sh 2>&1

# We need a blank line here for it to be a valid cron file

I can see the /etc/cron.d/backups-cron, but it's not executed. /pgenv.sh also exists, so /start.sh definitely has been executed.

So, the current master works for you without problems?

NyakudyaA commented 4 years ago

I haven't really tried with master for now. But I am using 11.0 with the 11.0 image. Can you share with me your full docker-compose or you are using the one in the repo. I can quickly try it out. I think

nilsnolde commented 4 years ago

Except for entrypoint override, I reverted back to the default one to test:

# This is for testing on travis only
# For production we recommend to rather use
# image: kartoza/pg-backup
# in the dbbackup service
db:
  image: kartoza/postgis
  environment:
    - ALLOW_IP_RANGE=0.0.0.0/0
    - POSTGRES_USER=docker
    - POSTGRES_PASS=docker

dbbackup:
  build: .
  entrypoint:
    - /bin/bash
    - /start.sh
  volumes:
    - ./backups:/backups
  environment:
    - DUMPPREFIX=watchkeeper
    - POSTGRES_HOST=db
    - POSTGRES_DBNAME=gis
    - POSTGRES_USER=docker
    - POSTGRES_PASS=docker
    - POSTGRES_PORT=5432
    # For restore script
    - TARGET_DB=gis
    - WITH_POSTGIS=1
    - TARGET_ARCHIVE=/backups/target_archive.dmp
  links:
    - db:db

I just realize that your postgis image is un-tagged. Shouldn't be a problem though, since the base image is still debian:buster, I guess.

Thanks so much for having a look. No post on SO or anywhere else suggests there's something wrong with the setup, I'm at a loss...

nilsnolde commented 4 years ago

oh wait, now I'm reading the comment about not being suitable for production!! Uff, maybe that's the reason huh.. Let me try again with a tagged image and see if that's the problem.

Sorry, that was stupid most likely..

nilsnolde commented 4 years ago

Hm, actually 11.0 is the same docker-compose.yml.. And it's not working for me either.

Gotta run now, but will get back to this when I come back home.

nilsnolde commented 4 years ago

Ok, I did a few more tests and got it working now. I had to change a couple of things though:

instead of placing the cron job to /etc/cron.d/, I registered it in start.sh just before cron -f with crontab /backups-cron, which places it in /var/spool/cron/root
in Dockerfile, I had to change the line ADD backups-cron /etc/cron.d/backups-cron to ADD backups-cron /backups-cron
I also placed start.sh as ENTRYPOINT instead of CMD, as it's really a container start-up command, not a build command and would get ignored if anyone would use any command-line --build-arg

I'm not sure why it wouldn't work for me and seemingly works for other people:

docker --version
Docker version 19.03.4, build 9013bf583a

docker-compose --version
docker-compose version 1.22.0, build f46880fe

Anyways, I can create a PR, first changing this, and then do my actual PR with some mods to env vars. Is that ok for you @NyakudyaA ?

NyakudyaA commented 4 years ago

Ok, I did a few more tests and got it working now. I had to change a couple of things though:
* instead of placing the cron job to `/etc/cron.d/`, I registered it in `start.sh` just before `cron -f` with `crontab /backups-cron`, which places it in `/var/spool/cron/root`

* in Dockerfile, I had to change the line `ADD backups-cron /etc/cron.d/backups-cron` to ` ADD backups-cron /backups-cron`

* I also placed `start.sh` as `ENTRYPOINT` instead of `CMD`, as it's really a container start-up command, not a build command and would get ignored if anyone would use any command-line `--build-arg`
I'm not sure why it wouldn't work for me and seemingly works for other people:
docker --version
Docker version 19.03.4, build 9013bf583a

docker-compose --version
docker-compose version 1.22.0, build f46880fe
Anyways, I can create a PR, first changing this, and then do my actual PR with some mods to env vars. Is that ok for you @NyakudyaA ?

Yes, that sounds fine. I will just review the logic you have done

kartoza / docker-pg-backup

Docker's cron job not working #24