Closed modem7 closed 1 year ago
The only downside I can see with supercronic is the multiarch aspect.
Installing it on a single arch is simple enough, but gets a bit more complicated with multi.
If we do go down this road, we may need to use someone else's work (eg https://github.com/hectorm/docker-supercronic) to copy across the relevant binary (https://github.com/hectorm/docker-supercronic/blob/master/Dockerfile.m4#L103) or do some fancy script work to determine the arch during build to grab the binaries (quite common in something like s6 builds, so we can probably find something relatively easily (eg https://gitlab.largenut.com/-/snippets/1) rather than reinvent the wheel).
Does not work for me ... :(
FROM python:3.10.6-alpine3.16 AS build
RUN apk add --no-cache go curl
RUN mkdir /supercronic
WORKDIR /supercronic
RUN curl -sSL https://github.com/aptible/supercronic/archive/refs/tags/v0.2.1.tar.gz | tar xz --strip 1
RUN go mod vendor && go install
FROM python:3.10.6-alpine3.16
COPY --from=build /root/go/bin/supercronic /usr/bin/supercronic
COPY *.sh /
RUN apk add --no-cache bash
CMD ["/usr/bin/supercronic", "-debug", "-passthrough-logs", "-split-logs", "/etc/borgmatic.d/crontab"]
*/2 * * * * /sleeper.sh
# sleeper.sh
#!/bin/bash
graceful_shutdown() {
echo Sleeper received $1 ... sleeping another 1s
sleep 1
echo Sleeper will exit now
exit 0
}
for s in SIGHUP SIGINT SIGTERM ; do
trap "graceful_shutdown $s" $s
done
while true ; do
echo Sleeping
sleep 10
done
time="2022-11-12T09:57:33Z" level=info msg="read crontab: /etc/borgmatic.d/crontab"
time="2022-11-12T09:57:33Z" level=debug msg="try parse (5 fields): '*/2 * * * *'"
time="2022-11-12T09:57:33Z" level=debug msg="job will run next at 2022-11-12 09:58:00 +0000 UTC" job.command=/sleeper.sh job.position=0 job.schedule="*/2 * * * *"
time="2022-11-12T09:58:00Z" level=info msg=starting iteration=0 job.command=/sleeper.sh job.position=0 job.schedule="*/2 * * * *"
Sleeping
Sleeping
invoke a docker stop
time="2022-11-12T09:58:30Z" level=info msg="received terminated, shutting down"
time="2022-11-12T09:58:30Z" level=info msg="waiting for jobs to finish"
Sleeping
$ echo $?
137
So looks like sleeper.sh does not receive the signal. Expected Result:
Sleeper received SIGTERM ... sleeping another 1s
Sleeper will exit now
Does not work for me ... :(
FROM python:3.10.6-alpine3.16 AS build RUN apk add --no-cache go curl RUN mkdir /supercronic WORKDIR /supercronic RUN curl -sSL https://github.com/aptible/supercronic/archive/refs/tags/v0.2.1.tar.gz | tar xz --strip 1 RUN go mod vendor && go install FROM python:3.10.6-alpine3.16 COPY --from=build /root/go/bin/supercronic /usr/bin/supercronic COPY *.sh / RUN apk add --no-cache bash CMD ["/usr/bin/supercronic", "-debug", "-passthrough-logs", "-split-logs", "/etc/borgmatic.d/crontab"]
*/2 * * * * /sleeper.sh
# sleeper.sh #!/bin/bash graceful_shutdown() { echo Sleeper received $1 ... sleeping another 1s sleep 1 echo Sleeper will exit now exit 0 } for s in SIGHUP SIGINT SIGTERM ; do trap "graceful_shutdown $s" $s done while true ; do echo Sleeping sleep 10 done
time="2022-11-12T09:57:33Z" level=info msg="read crontab: /etc/borgmatic.d/crontab" time="2022-11-12T09:57:33Z" level=debug msg="try parse (5 fields): '*/2 * * * *'" time="2022-11-12T09:57:33Z" level=debug msg="job will run next at 2022-11-12 09:58:00 +0000 UTC" job.command=/sleeper.sh job.position=0 job.schedule="*/2 * * * *" time="2022-11-12T09:58:00Z" level=info msg=starting iteration=0 job.command=/sleeper.sh job.position=0 job.schedule="*/2 * * * *" Sleeping Sleeping
invoke a
docker stop
time="2022-11-12T09:58:30Z" level=info msg="received terminated, shutting down" time="2022-11-12T09:58:30Z" level=info msg="waiting for jobs to finish" Sleeping $ echo $? 137
So looks like sleeper.sh does not receive the signal. Expected Result:
Sleeper received SIGTERM ... sleeping another 1s Sleeper will exit now
I suspect that tini/dumb-init (or similar) is still required in this case. It might also be worthwhile to check with docker top what's actually going on inside the container with your particular dockerfile + scripts.
docker top containername x -o pid,command --forest
I suspect that tini/dumb-init (or similar) is still required in this case. It might also be worthwhile to check with docker top what's actually going on inside the container with your particular dockerfile + scripts.
docker top containername x -o pid,command --forest
I did a quick test with tini, dumb-init and supervisord.
Only supervisord seemed to forward the signal properly.
Neither Cron nor Supercronic forwarded the signal to the process it calls.
So next to test if borgmatic signals its hook scripts... 😉
So next to test if borgmatic signals its hook scripts... 😉
We may need to work with the supercronic guys on it (I still think the benefits of supercronic are worthwhile, we'll figure out how to do the multiarch builds for it later) if the hook scripts don't send a signal.
@toastie89 and/or @witten may be the better people to ask regarding the signals however, as I don't know the specifics.
I believe (although I haven't confirmed) that borgmatic passes the following signals through to any hook processes: SIGHUP, SIGTERM, SIGUSR1, and SIGUSR2.
I believe (although I haven't confirmed) that borgmatic passes the following signals through to any hook processes: SIGHUP, SIGTERM, SIGUSR1, and SIGUSR2.
Than just forget about all the init stuff and use my proposal to signal first borgmatic and then crond, 😄
Right now I don't have an idea of what problems we are not solving with this approach
Thinking out side the box on this, @witten, instead of us using cron within the container would it be possible to add some sort of
on:
schedule: 30 0 * * *
config to Borgmatic and we run borgmatic --scheduler
or something as PID 1 and this solves all problems
Thinking out side the box on this, @witten, instead of us using cron within the container would it be possible to add some sort of
on: schedule: 30 0 * * *
config to Borgmatic and we run
borgmatic --scheduler
or something as PID 1 and this solves all problems
Imo, we'd probably still need an entrypoint script due to our variables etc, however, agreed. It would make things a lot simpler as cron isn't great.
Thinking out side the box on this, @witten, instead of us using cron within the container would it be possible to add some sort of
on: schedule: 30 0 * * *
config to Borgmatic and we run
borgmatic --scheduler
or something as PID 1 and this solves all problems
Imo, we'd probably still need an entrypoint script due to our variables etc, however, agreed. It would make things a lot simpler as cron isn't great.
But we could have the ENTRYPOINT
as the script, then the CMD
as borgmatic so it gets PID1.
But we could have the ENTRYPOINT as the script, then the CMD as borgmatic so it gets PID1.
Thats easy. Do your stuff in entry.sh
and than exec borgmatic --scheduler
.
Thinking out side the box on this, @witten, instead of us using cron within the container would it be possible to add some sort of
on: schedule: 30 0 * * *
config to Borgmatic and we run
borgmatic --scheduler
or something as PID 1 and this solves all problems
In theory, that would be possible. But I was also hoping not to reinvent cron inside borgmatic. :smiley:
EDIT: Relevant borgmatic ticket: https://projects.torsion.org/borgmatic-collective/borgmatic/issues/616
@modem7 - We should be able to build supercronic per arch by adding another staged build process of something like this:
FROM golang:alpine AS gobuilder
ARG SUPERCRONIC_VERSION="v0.2.1"
RUN apk add --no-cache git
RUN go install github.com/aptible/supercronic@"${SUPERCRONIC_VERSION}"
# The in the borgmatic build stage
COPY --from=gobuilder /go/bin/supercronic /usr/local/bin/
I just tested this locally (had to remove --link on the COPY as docker for mac didn't like it) and it builds supercronic that way. This would give us multi arch capabilities. I also like supercronic has a -test function we could implement on start up so if someone has a dodgy crontab we could exit 1
Continuing conversation from https://github.com/borgmatic-collective/docker-borgmatic/pull/173.
Is Alpine cron really the best tool to use here I wonder?
I just found supercronic which seems like a potentially better tool than cron itself, and more suited to a container environment.
It also supports more traditional cron notations such as
@hourly
and\
giving our users finer control over their cron timings.Also, given how the logging works with supercronic, it may also resolve https://github.com/borgmatic-collective/docker-borgmatic/pull/174 without additional changes as a side effect.