u1735067 / bbbs

Borg wrappers to handle pull-mode operation. Also comes with installer and ssh key provisioning.
BSD 3-Clause "New" or "Revised" License
11 stars 1 forks source link

Bash Borg Backup System (BBBS)

Borg wrappers for server pull mode. As distributions repositories may not always be updated, it is designed to work with the standalone version, installed in /opt/bbbs.

Features

Limitations

Requirement

Instructions

One option is to git clone this repo in /opt/bbbs/ and run the installer.\ Obviously, you have to create do-backup from do-backup.example according to your needs and mark it as executable (chmod +x /opt/bbbs/do-backup).\ borg's dot path are symlinked to more accessible paths (.cache/borg -> ~/cache, .config/borg -> ~/ (to expose keys and security directly), .ssh -> ssh).

client> apt-get update && apt-get install git sudo socat curl jq
client> git clone https://github.com/u1735067/bbbs.git /opt/bbbs
client> /opt/bbbs/installer.sh install-client

server> yum install git sudo socat curl jq
server> git clone https://github.com/u1735067/bbbs.git /opt/bbbs
server> /opt/bbbs/installer.sh install-server

server> /opt/bbbs/ssh-gen-copy-key --generate --key-name server-name_hypervisor --copy --user borg -- root@server-name -p 22
server> gw_key=~borg/ssh/key_server-name_hypervisor; /opt/bbbs/ssh-gen-copy-key --generate --key-name server-name_vm-1 --copy --user borg -- root@172.16.0.1 -p 22 -o ProxyCommand="ssh -i$gw_key -W %h:%p server-name.fqdn -p 22"

server> /opt/bbbs/borg init -e authenticated-blake2 /srv/borg/server-name/hypervisor
server> /opt/bbbs/borg init -e authenticated-blake2 /srv/borg/server-name/vm-1

server> sudo -u borg /opt/bbbs/do-backup

server> /opt/bbbs/borg info /srv/borg/server-name/hypervisor

bbbs-client will call ~borg/backup-pre and ~borg/backup-pre on the client before and after running borg create, you can use them to dump a sql database and remove it for example.

To manage ssh connexion parameters, you can also use ssh_config instead, for example:

server> cat ~borg/ssh/config
Host *
    User borg

Host server-name
    Port 22
    IdentityFile ~borg/ssh/key_server-name_hypervisor

Host 172.16.*.*
    ProxyCommand ssh -F ~borg/ssh/config -W %h:%p server-name

Diagram

╔══════╦════════════════════════════════════════════════════╦══════════════════════════════════════════════════════════════╗
║      ║                       Server                       ║                            Client                            ║
╠══════╬════════════════════════════════════════════════════╬══════════════════════════════════════════════════════════════╣
║ root ║  sudo                                              ║            bbbs-client ─┬─► ~borg/backup-pre                 ║
║      ║    │                                               ║              ▲          ├─► borg create ─► wrapper ─► socat  ║
║      ║    │                                               ║              │          └─► ~borg/backup-post           │    ║
║      ║    │                                               ║              │                                          │    ║
╠══════╬════┼═══════════════════════════════════════════════╬══════════════┼══════════════════════════════════════════╪════╣
║ borg ║    └─► do-backup                                   ║              │                                          │    ║
║      ║             └─► wrapper                            ║              │                                          │    ║
║      ║                   ├──────────────────────► ssh ====╬===► ssh ──► sudo                                        │    ║
║      ║                   └─► socat ──► borg serve  ┊      ║      ┊                                                  │    ║
║      ║                          ▲                  ┊      ║      ┊                                                  │    ║
║      ║                          └──────────────────╘======╬======╛◄─────────────────────────────────────────────────┘    ║
╚══════╩════════════════════════════════════════════════════╩══════════════════════════════════════════════════════════════╝
  ===== = ssh channel

Pre & post hooks on client

*/!\ Don't forget to set executable bit (`chmod +x ~borg/backup-p`) on them!**

Mysql

Cache credentials in ~borg/.mylogin.cnf (see mysqldump, mysql_config_editor):

su borg -c "mysql_config_editor set --user=root --password"

or for MariaDB in ~borg/.my.cnf (see mysqldump):

[mysqldump]
user=mysqluser
password=secret

~borg/backup-pre:

#!/usr/bin/env bash

echo "- Dumping MySQL ..."
mkdir --parents /var/backups/mysql
# GZip directly in case the database is huge -- could be handled by borg using stdin maybe
mysqldump --all-databases --single-transaction \
    | gzip --fast --rsyncable \
    > /var/backups/mysql/borg-dump_$(date --utc "+%Y-%m-%d_%H.%M.%SZ").sql.gz
echo "- Dumping MySQL ... Done"

~borg/backup-post:

#!/usr/bin/env bash

echo "- Removing MySQL Dump ..."
rm -f /var/backups/mysql/borg-*
echo "- Removing MySQL Dump ... Done"

Splunk

See Backup indexed data, Index backup strategy.

~borg/backup-pre:

#!/usr/bin/env bash

echo "- Dumping Splunk ..."
mkdir --parents /var/backups/splunk
/opt/splunk/bin/splunk search "index=*" -output json -maxout 0 \
    | gzip --fast --rsyncable \
    > /var/backups/splunk/borg-dump_$(date --utc "+%Y-%m-%d_%H.%M.%SZ").json.gz
echo "- Dumping Splunk ... Done"

echo "- Stopping Splunk ..."
service splunk stop
#systemctl stop splunk
echo "- Stopping Splunk ... Done"

~borg/backup-post:

#!/usr/bin/env bash

echo "- Removing Splunk Dump ..."
rm -f /var/backups/splunk/borg-*
echo "- Removing Splunk Dump ... Done"

echo "- Starting Splunk ..."
service splunk start # || mail failed to restart service, check ASAP
#systemctl start splunk
echo "- Starting Splunk ... Done"

Why?

While Borg Backup Server (BBS) has been announced, it is not available yet and I needed a solution (preferably before my server crashes). Also, I don't want the clients to have to know how to reach the server. This is ugly but probably the simplest for now (when using pull mode). A correct solution could be to use python as wrapper, with libssh to setup the channel, and parsing nice configuration files.

Locale correction explanation

The wrappers borg and bbbs-client will try to set a correct locale (meaning: with UTF-8 charset).

Why?

Because Python relies on it to read filenames correctly. Most Unix filesystems seems to only care about bytes, but the convention is to use UTF-8 and most locales without the charset specifier (.utf8) doesn't not use UTF-8, so Python use that charset and fails to decode filenames properly. Also, distributions like OMV can decide to set LANG=C before executing crons, which will make Python use the ascii/ANSI encoding.

However, except for extract / fuse mouse according to the issues, borg seems to handle that correctly (even though I'm not sure why, as args.path are str, so scandir will return str which means they are decoded?).

More informations (about this mess):

How?

To do so, it'll parse locales by priority (LC_ALL > LC_CTYPE > LANG), testing if it's indicating UTF-8 charset, or if it's not trying to find the UTF-8 alternative of this locale, or ultimately unsetting it (to let the chance to lower priority locales).\ If there's no locale left, it'll then try to see if there's any UTF-8 locale available, setting LC_CTYPE, starting by C, English (en_GB, en_US, en_*) and ultimatly the first found.\ In the end, if there's really no UTF-8 locale available, none of LC_ALL, LC_CTYPE, LANG should be set, thus triggering the python 3 default UTF-8, but I wasn't able to reproduce this behavior:

# export -n LC_ALL LC_CTYPE LANG; python3 -c 'import sys; print(sys.version); print(sys.getfilesystemencoding())'
3.4.2 (default, Oct  8 2014, 10:45:20)
[GCC 4.9.1]
ascii