[FEATURE] providing rsync for backups

[rdxmb]

Hello,

with an increasing instance of OTRS over years, creating backups through tar in our case is not very efficient. Additionally, running tar with zip twice takes very long, but does not really reduce space.

I am going to write a single rsync-job for our environment to run rsync. The backup.pl will only be used for creating the dumps then.

Are you interested in a PR to also use that? I need two things:

1. run yum install rsync in the Dockerfile. I'd like to add this in any case - no matter if we add the rsync-script in this repository or not. Ok?
2. I think about a separate file /otrs_backup_rsync.sh file, because the /otrs_backup.sh has some kind of overkill I do not want to handle with (see above or my comment in https://github.com/juanluisbaptiste/docker-otrs/issues/96#issuecomment-664469029) . We could also move the content of otrs_backup.sh to otrs_backup-tar.sh - and use backup.sh as a wrapper script, together with a variable containing tar or rsync - and trigger backup-tar.sh or backup-rsync.sh with that. This way, we can keep this backwards compatible.

What do you think?

EDIT: If you do not want this, I will just mount a simple /otrs_backup.sh in my environment to keep it simple :)

[rdxmb]

@juanluisbaptiste what do you think?

[rdxmb]

changing this in my environment. Closing this.

[juanluisbaptiste]

@rdxmb sorry for the late reply, I needed to have some time to review this and think about the implications before answering.

I think running rsync should be run from outside the OTRS container, it is not the container's responsibility to make remote copies of the backup files. An external process should take care of this, copying the backup files from the host directory of the backups volume to the remote location using rsync. Also, implementing this would mean adding many options to configure the rsync process, adding complexity to the container configuration.

What could be done is to add an option to control the creation of the final backup file that contains the backup tarballs created by the OTRS backup script.

[rdxmb]

@juanluisbaptiste

thanks for your answer.

In my case, the rsync replaces the tar.

My tar-backup to NFS was running 4 hours each night. A test to restore needed about 10 hours. That's quite too long - considered that the diff is very small and it is not neccessary to tar all the articles from years every night.

So I run rsync as the first backup inside the container (instead of tar). This allows to start and stop the application like you do in otrs_backup.sh . With rsync, the backup of the application data runs about 20min and copies only a small amount of files.

This daily full backup (comparable to your tarball) is copied via an external job (https://restic.net/ in my case) to external space to have the history. (Restic creates an instance per night, called snaphots). This job runs, of course, outside of the application container.

There is no problem for me with doing this by a mount of my own otrs_backup.sh. However, having rsync in the container would be great. Actually, the backup-script has to install it first. That's not the best way to get the binary.

[juanluisbaptiste]

I still do not understand why do you need to run rsync from inside the container if you have the data that is backed up outside of it.

[rdxmb]

What do you mean with "outside of it"?

My otrs data is mounted via iSCSI into the container. This looks like

/dev/sdd on /opt/otrs/db_upgrade type ext4 (rw,relatime)
/dev/sdd on /opt/otrs/addons type ext4 (rw,relatime)
/dev/sdd on /opt/otrs/Kernel type ext4 (rw,relatime)
/dev/sdd on /opt/otrs/var/article type ext4 (rw,relatime)
/dev/sdd on /opt/otrs/var/httpd/htdocs/skins type ext4 (rw,relatime)

The backup space is nfs and is mounted to /var/otrs/backups

To get a consistent backup of my otrs instance, I run my own otrs_backup.sh, which lookes like this (not finished for production usage)

# --- snip

yum install -y rsync  # missing in the Dockerfile
stop_all_services

rsync "--exclude=var/tmp --exclude=js-cache --exclude=css-cache --exclude=.git" -ai --delete-after --delete-excluded --backup --backup-dir /var/otrs/backups/filesystem/diff_before_latest/ /opt/otrs/ /var/otrs/backups/filesystem/ 
/opt/otrs/scripts/backup.pl -d ${OTRS_BACKUP_DIR}/dump -t [mark]dbonly[/mark] -r 2 -c gzip

start_all_services

# --- snip

I do not see a solution to do that outside of the container.