Move to S6 switched crond to busybox instead of supercronic.

[jinnatar]

Supercronic is not getting run anymore and instead it's busybox crond. Documentation still seems to point towards using supercronic so I'm assuming this is a bug of some sort?

%> docker run --rm -it ghcr.io/borgmatic-collective/borgmatic:latest
Unable to find image 'ghcr.io/borgmatic-collective/borgmatic:latest' locally
latest: Pulling from borgmatic-collective/borgmatic
Digest: sha256:d6f929033c1a7b951c01f77a4393ff7740a8f58579a0a12b2fc53f8cb7e1ce81
Status: Downloaded newer image for ghcr.io/borgmatic-collective/borgmatic:latest
Software Versions:
-----------------------------------
apprise 1.7.6
borg 1.2.8
borgmatic 1.8.10
dockercli not installed
python 3.12.3
-----------------------------------
Time Zone: Europe/London
-----------------------------------

Applying crontab.txt
crontab: can't open '/etc/borgmatic.d/crontab.txt': No such file or directory
Cron job set as: 
# do daily/weekly/monthly maintenance
# min   hour    day     month   weekday command
*/15    *       *       *       *       run-parts /etc/periodic/15min
0       *       *       *       *       run-parts /etc/periodic/hourly
0       2       *       *       *       run-parts /etc/periodic/daily
0       3       *       *       6       run-parts /etc/periodic/weekly
0       5       1       *       *       run-parts /etc/periodic/monthly
crond: crond (busybox 1.36.1) started, log level 8

[jinnatar]

It seems that commit 670f70eabbdc216518811d5b6e9a1607db9eda69 is the trigger. entry.sh is now dead code and never run, and instead the s6 svc-cron module is used which just launches crond.

[MarkJonas]

I consider it a bug that supercronic was accidentally (?!) removed.

My error output looks like this:

Applying crontab.txt
Cron job set as: 
# Backup hourly from 8:00 to 22:00
./run: line 112: printf: `Y': invalid format character
0 8-22/1 * * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-`date +crond: crond (busybox 1.36.1) started, log level 8

[grantbevis]

@modem7 did we mean to strip supercronic out?

[modem7]

@grantbevis yes, the only reason supercronic was in was to address the sigterm issue which it didn't really do anyway.

Since moving to s6, it's less maintenence and more reliable to remove supercronic and use the inbuilt cron system.

We need to update and go through the readme and update since we're missing a few things.

[MarkJonas]

We need to update and go through the readme and update since we're missing a few things.

A migration guide would be great because it seems that the cron syntax is different.

[modem7]

We need to update and go through the readme and update since we're missing a few things.

A migration guide would be great because it seems that the cron syntax is different.

The main difference between cron and supercronic is that supercronic allows for second resolution schedules as it uses the cronexpr package (so it accepts seconds and years), giving a 6-7 digit expression (2 of which are optional), whereas standard cron is typically 5 digits (minutes, hours, days, month, day of week).

Field name     Mandatory?   Allowed values    Allowed special characters
----------     ----------   --------------    --------------------------
Seconds        No           0-59              * / , -
Minutes        Yes          0-59              * / , -
Hours          Yes          0-23              * / , -
Day of month   Yes          1-31              * / , - L W
Month          Yes          1-12 or JAN-DEC   * / , -
Day of week    Yes          0-6 or SUN-SAT    * / , - L #
Year           No           1970–2099         * / , -

To migrate, just make it first resolution (5 digits, instead of 6 or 7).

Using something like crobtab guru is useful in this endeavour.

[grantbevis]

Thanks bro @modem7

[MarkJonas]

It seems it is not that easy to move an existing crontab to s6. I changed the number of fields from 6 to 5 but yet I get a strange error at startup.

Software Versions:
-----------------------------------
apprise 1.7.6
borg 1.2.8
borgmatic 1.8.10
dockercli not installed
python 3.12.3
-----------------------------------
Time Zone: Europe/Berlin
-----------------------------------
Applying crontab.txt
Cron job set as: 
# Backup hourly from 8:00 to 22:00
./run: line 112: printf: `Y': invalid format character
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-`date +crond: crond (busybox 1.36.1) started, log level 8

crontab.txt:

# Backup hourly from 8:00 to 22:00
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-`date +%Y-%m-%dT%H:%M:%S`.log
# Check once a day at midnight
@daily  PATH=$PATH:/usr/bin borgmatic check -v 1 2>&1 | tee /mnt/log/check-`date +%Y-%m-%dT%H:%M:%S`.log

Only after stripping down the crontab to the following reduced version it worked again.

0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1

Applying crontab.txt
Cron job set as: 
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1
crond: crond (busybox 1.36.1) started, log level 8

I suspect that something does parse anymore in combination with the date command (the only thing containing a Y). But I am clueless what the problem really is.

Help to get the crontab working again would be highly appreciated. :)

[modem7]

It seems it is not that easy to move an existing crontab to s6. I changed the number of fields from 6 to 5 but yet I get a strange error at startup.

Software Versions:
-----------------------------------
apprise 1.7.6
borg 1.2.8
borgmatic 1.8.10
dockercli not installed
python 3.12.3
-----------------------------------
Time Zone: Europe/Berlin
-----------------------------------
Applying crontab.txt
Cron job set as: 
# Backup hourly from 8:00 to 22:00
./run: line 112: printf: `Y': invalid format character
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-`date +crond: crond (busybox 1.36.1) started, log level 8

crontab.txt:

# Backup hourly from 8:00 to 22:00
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-`date +%Y-%m-%dT%H:%M:%S`.log
# Check once a day at midnight
@daily  PATH=$PATH:/usr/bin borgmatic check -v 1 2>&1 | tee /mnt/log/check-`date +%Y-%m-%dT%H:%M:%S`.log

Only after stripping down the crontab to the following reduced version it worked again.

0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1

Applying crontab.txt
Cron job set as: 
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1
crond: crond (busybox 1.36.1) started, log level 8

I suspect that something does parse anymore in combination with the date command (the only thing containing a Y). But I am clueless what the problem really is.

Help to get the crontab working again would be highly appreciated. :)

Could you paste your compose file? When I'm back home I'll take a look .

[MarkJonas]

That's my compose file. I only cut down the backup source paths.

services:
  borgmatic:
    image: b3vis/borgmatic:1.8.10
    container_name: borgmatic
    hostname: Borgmatic # important so {hostname} in Borgmatic config works like expected
    environment:
      - TZ=Europe/Berlin
      - BORG_PASSPHRASE=${BORG_PASSPHRASE} # set inside Portainer
    volumes:
      # Borg Backup / Borgmatic
      - /volume1/docker/borgmatic/borgmatic.d:/etc/borgmatic.d # borgmatic config file(s) + crontab.txt
      - /volume1/docker/borgmatic/.borgmatic:/root/.borgmatic  # borgmatic state files
      - /volume1/docker/borgmatic/config:/root/.config/borg    # config and keyfiles
      - /volume1/docker/borgmatic/ssh:/root/.ssh               # ssh key for remote repositories
      - /volume1/docker/borgmatic/cache:/root/.cache/borg      # checksums used for deduplication
      - /volume1/docker/borgmatic/log:/mnt/log                 # log file storage
      # Backup target
      - /volumeUSB1/usbshare/borg-backup-diskstation:/mnt/borg-repository
      # Backup sources
      - /volume2/foo:/mnt/source/foo:ro
      - /volume2/bar:/mnt/source/bar:ro
    restart: always

[modem7]

OK, I think I've found the issue.

I'm going to raise a PR, and we'll need to change your crontab as well to work a bit better.

[modem7]

I've raised https://github.com/borgmatic-collective/docker-borgmatic/pull/322. @grantbevis could you review and authorise? Single line change.

To make this work better, change the crontab to be:

# Backup hourly from 8:00 to 22:00
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-$(date +%Y-%m-%dT%H:%M:%S).log
# Check once a day at midnight
@daily  PATH=$PATH:/usr/bin borgmatic check -v 1 2>&1 | tee /mnt/log/check-$(date +%Y-%m-%dT%H:%M:%S).log

Better yet, in the new build, you can even have this as env vars, fully eliminating your crontab: Note the CRON, CRON_COMMAND and EXTRA_CRON variables.

  borgmatic:
    # image: modem7/borgmatic-docker
    image: borgmatic:test
    container_name: Borgmatic
    environment:
      TZ: $TZ
      BORG_PASSPHRASE: $BORG_PASSPHRASE
      BORG_SOURCE_1: $BORG_SOURCE_1
      BORG_SOURCE_2: $BORG_SOURCE_2
      BORG_REPO: $BORG_REPO
      BORG_HEALTHCHECK_URL: $BORG_HEALTHCHECK_URL
      CRON: "0 8-22/1 * * *"
      DOCKERCLI: true
      CRON_COMMAND: "borgmatic prune compact create --files --stats -v 1 2>&1 > tee /mnt/log/backup-$(date +%Y-%m-%dT%H:%M:%S).log"
      EXTRA_CRON: "@daily borgmatic check -v 1 2>&1 > tee /mnt/log/check-$(date +%Y-%m-%dT%H:%M:%S).log"

Once the PR is merged, this is the result I'm getting:

Installing Docker CLI...
Software Versions:
-----------------------------------
apprise 1.7.6
borg 1.2.8
borgmatic 1.8.10
dockercli 25.0.5
python 3.12.3
-----------------------------------
Time Zone: Europe/London
-----------------------------------

Applying custom cron
Cron job set as: 
0 8-22/1 * * * borgmatic prune compact create --files --stats -v 1 2>&1 > tee /mnt/log/backup-$(date +%Y-%m-%dT%H:%M:%S).log
@daily borgmatic check -v 1 2>&1 > tee /mnt/log/check-$(date +%Y-%m-%dT%H:%M:%S).log

crond: crond (busybox 1.36.1) started, log level 8

[grantbevis]

https://github.com/borgmatic-collective/docker-borgmatic/pull/322 Is merged

[modem7]

@MarkJonas - Does this work for you?

[MarkJonas]

@modem7 Thank you for your help.

I tried the modified crontab.txt (replace backticks with $()) and it still does not work with 1.8.10.

Software Versions:
-----------------------------------
apprise 1.7.6
borg 1.2.8
borgmatic 1.8.10
dockercli not installed
python 3.12.3
-----------------------------------
Time Zone: Europe/Berlin
-----------------------------------
Applying crontab.txt
Cron job set as: 
# Backup hourly from 8:00 to 22:00
./run: line 112: printf: `Y': invalid format character
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-$(date +crond: crond (busybox 1.36.1) started, log level 8

Next, I switched to latest and there it works.

[custom-init] Docker CLI variable not set, skipping...
[custom-init] No custom packages found, skipping...
-----------------------------------
Software Versions:
-----------------------------------
apprise 1.7.6
borg 1.2.8
borgmatic 1.8.11
dockercli not installed
composecli not installed
python 3.12.3
-----------------------------------
Time Zone: Europe/Berlin
-----------------------------------
Applying crontab.txt
Cron job set as: 
# Backup hourly from 8:00 to 22:00
0 8-22/1 * * * PATH=$PATH:/usr/bin borgmatic prune compact create --files --stats -v 1 2>&1 | tee /mnt/log/backup-$(date +%Y-%m-%dT%H:%M:%S).log
# Check once a day at midnight
@daily  PATH=$PATH:/usr/bin borgmatic check -v 1 2>&1 | tee /mnt/log/check-$(date +%Y-%m-%dT%H:%M:%S).log
crond: crond (busybox 1.36.1) started, log level 8

Thank you very much! :+1:

I now want to scan through the documentation and read more about your proposal to remove the crontab.txt and use the environment parameters instead.

At a quick glance I did not find documentation e.g. for CRON_COMMAND. I just found its use in root/etc/s6-overlay/s6-rc.d/svc-cron/run. There, the use of CRON seems to be a mismatch compared to the description of BACKUP_CRON in README.md.

[modem7]

Heya,

Yes, the readme needs an overhaul as we've have a lot of changes in a short period of time.

But effectively, it merely transposes cron from a file to a variable which can be much more easily changed.

The cron logic in run is https://github.com/borgmatic-collective/docker-borgmatic/blob/be6bf729fda7e257b8c600a7916de50433db30ad/root/etc/s6-overlay/s6-rc.d/svc-cron/run#L90-L113

The downside is that multiple line cron jobs can get complicated.