willfarrell / docker-crontab

A docker job scheduler (aka. crontab for docker)
MIT License
300 stars 65 forks source link

Logs not written to file #41

Open danobot opened 3 years ago

danobot commented 3 years ago

Hey! Love this project, exactly what I needed. Got it mostly set up now but facing an issue with the jobs.log not getting written to and some strange errors in the logs.

 jq: error (at <stdin>:1): Cannot iterate over null (null)

Docker Compose File

version: "3.7"

services:
  ftp_mount:
    image: ubuntu
    restart: always
    command: "sh -c 'while :; do sleep 1; done'"
    volumes:
      - "/mnt/disks/Unprotected_Data/FTP/reolink:/ftp"
      - "/mnt/disks/ntfs_backup_drive_500gb/BACKUPS_CRONTAB:/backups"
      - "/mnt/user/appdata/:/data:ro"
      - "/mnt/user/docker/:/docker:ro"
      - "${PWD}/scripts:/scripts:Z"
      - "${PWD}/logs:/var/log/crontab:rw"

  crontab:
    build: .
    restart: always
    volumes:
     - "/var/run/docker.sock:/var/run/docker.sock:ro"
     - "${PWD}/config/config.yaml:/opt/crontab/config.yaml:ro"
     - "${PWD}/logs:/var/log/crontab:rw"

config.yaml

purge_camera_files:
  command: echo Purging Camera Files
  comment: Removes camera footage until disk is 95% full
  image: alpine:3.5
  schedule: '@every 24h'
  dockerargs: --rm
  trigger:
    - command: /bin/bash /scripts/purge_files.sh /ftp
      project: crontab
      container: ftp_mount

Container Logs:

root@Tower:/mnt/user/docker/crontab# coml
Attaching to crontab_ftp_mount_1, crontab_crontab_1
crontab_1    | 2021-01-01T06:09:49.378156977Z jq: error (at <stdin>:1): Cannot iterate over null (null)
crontab_1    | 2021-01-01T06:09:49.416839397Z jq: error (at <stdin>:1): Cannot iterate over null (null)
crontab_1    | 2021-01-01T06:09:49.454677066Z jq: error (at <stdin>:1): Cannot iterate over null (null)
crontab_1    | 2021-01-01T06:09:49.566032634Z jq: error (at <stdin>:1): Cannot iterate over null (null)
crontab_1    | 2021-01-01T06:09:50.083452378Z ##### crontab generation complete #####
crontab_1    | 2021-01-01T06:09:50.083854115Z # Removes camera footage until disk is 95% full
crontab_1    | 2021-01-01T06:09:50.083868869Z */1 * * * * /bin/bash /opt/crontab/jobs/548a081a-c6ec-4db1-b270-82434eda680a.sh
crontab_1    | 2021-01-01T06:09:50.083920757Z ##### run commands with onstart #####
crontab_1    | 2021-01-01T06:09:50.083968699Z crond -f -d 6 -c /etc/crontabs
crontab_1    | 2021-01-01T06:09:50.084371880Z crond: crond (busybox 1.31.1) started, log level 6
crontab_1    | 2021-01-01T06:10:00.084930783Z crond: USER docker pid 135 cmd /bin/bash /opt/crontab/jobs/548a081a-c6ec-4db1-b270-82434eda680a.sh
crontab_1    | 2021-01-01T06:10:00.085997105Z Start Cronjob **548a081a-c6ec-4db1-b270-82434eda680a** Removes camera footage until disk is 95% full
crontab_1    | 2021-01-01T06:10:02.233314641Z Purging Camera Files
crontab_1    | 2021-01-01T06:10:04.056799272Z Current capacity at 91 %
crontab_1    | 2021-01-01T06:10:04.072716733Z End Cronjob **548a081a-c6ec-4db1-b270-82434eda680a** Removes camera footage until disk is 95% full
willfarrell commented 3 years ago

Can you try with v0.6, see if that works.

danobot commented 3 years ago

That doesn't work either.

Docker compose:


  crontab:
    # build: .
    image: willfarrell/crontab:0.6.0
    restart: always
    volumes:
     - "/var/run/docker.sock:/var/run/docker.sock:ro"
     - "${PWD}/config/config.yaml:/opt/crontab/config.yaml:ro"
     - "${PWD}/config-samples/config.sample.json:/opt/crontab/config.json:ro"
     - "${PWD}/logs:/var/log/crontab:rw"

Config.json

[
  {
    "comment": "map a volume",
    "schedule": "* * * * *",
    "dockerargs": "-d -v /tmp:/tmp",
    "command": "echo new",
    "image": "alpine:3.5"
  }
]
crontab_1    | 2021-01-01T10:41:21.552122125Z ##### crontab generation complete #####
crontab_1    | 2021-01-01T10:41:21.552632617Z # map a volume
crontab_1    | 2021-01-01T10:41:21.552646044Z * * * * * /bin/bash /opt/crontab/jobs/a37a44d3-fa7e-42a3-8067-e15dc0fe0273.sh
crontab_1    | 2021-01-01T10:41:21.552692428Z ##### run commands with onstart #####
crontab_1    | 2021-01-01T10:41:21.552767279Z crond -f -d 6 -c /etc/crontabs
crontab_1    | 2021-01-01T10:41:21.553190399Z crond: crond (busybox 1.31.1) started, log level 6
crontab_1    | 2021-01-01T10:42:00.553702943Z crond: USER docker pid  98 cmd /bin/bash /opt/crontab/jobs/a37a44d3-fa7e-42a3-8067-e15dc0fe0273.sh
crontab_1    | 2021-01-01T10:42:00.554789771Z Start Cronjob **a37a44d3-fa7e-42a3-8067-e15dc0fe0273** map a volume
crontab_1    | 2021-01-01T10:42:00.908710241Z 1f1b7b4b527ba926b4c9b33e1d7cbea932ebcd49190134db81db397904448bd5
crontab_1    | 2021-01-01T10:42:02.317511655Z End Cronjob **a37a44d3-fa7e-42a3-8067-e15dc0fe0273** map a volume
crontab_1    | 2021-01-01T10:43:00.554039511Z crond: USER docker pid 189 cmd /bin/bash /opt/crontab/jobs/a37a44d3-fa7e-42a3-8067-e15dc0fe0273.sh
crontab_1    | 2021-01-01T10:43:00.555075296Z Start Cronjob **a37a44d3-fa7e-42a3-8067-e15dc0fe0273** map a volume
crontab_1    | 2021-01-01T10:43:01.755776060Z dd678351875a1fd595a1460bd28f24fc6a6bfb97cfeffb5e9a1d2b174d8434ca
crontab_1    | 2021-01-01T10:43:04.093320629Z End Cronjob **a37a44d3-fa7e-42a3-8067-e15dc0fe0273** map a volume

The volume is mapped correctly and I can see the jobs.log file is getting created, but it is an empty file

KinGsNowGH commented 3 years ago

Same problem:

docker-compose:

  crontab01:
    image: willfarrell/crontab:1.0.0
    container_name: crontab01
    # environment:
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # - ./env:/opt/env:ro
      # - ./docker-compose-crontab/opt/crontab:/opt/crontab:rw
      - ./docker-compose-crontab/opt/crontab/config.yml:/opt/crontab/config.yml:rw
      # - ./crontab01-docker/var/log/crontab:/var/log/crontab:rw
    restart: always
    # healthcheck:
    network_mode: none
    # networks:
    # command:

config.yml

redmine_plugin_recurring_task:
  command: sh -c 'cd /home/redmine/redmine && RAILS_ENV=production bundle exec rake redmine_recurring_tasks:exec'
  comment: redmine_plugin_recurring_task
  container: redmine01
  # project: docker-compose-redmine
  schedule: '@every 1m'

Logs

2021-02-03T13:20:24.664390515Z ##### crontab generation complete #####
2021-02-03T13:20:24.665872468Z # redmine_plugin_recurring_task
2021-02-03T13:20:24.665918845Z */1 * * * * /bin/bash /opt/crontab/jobs/810847fb-441a-439c-9885-e3c04e59e7e9.sh
2021-02-03T13:20:24.666450460Z ##### run commands with onstart #####
2021-02-03T13:20:24.666786865Z crond -f -d 6 -c /etc/crontabs
2021-02-03T13:20:24.668117210Z crond: crond (busybox 1.31.1) started, log level 6
2021-02-03T13:21:00.670243017Z crond: USER docker pid 138 cmd /bin/bash /opt/crontab/jobs/810847fb-441a-439c-9885-e3c04e59e7e9.sh
2021-02-03T13:21:00.674956752Z Start Cronjob **810847fb-441a-439c-9885-e3c04e59e7e9** redmine_plugin_recurring_task
2021-02-03T13:21:12.226642170Z End Cronjob **810847fb-441a-439c-9885-e3c04e59e7e9** redmine_plugin_recurring_task

I tought i had a problem with volumes, but inside the container i still dont get any log:

bash-5.0# pwd
/var/log/crontab
bash-5.0# cat jobs.log
bash-5.0#