This module is intended to supplement the configuration of a Bareos
(or Bacula) server with the "dynamic" bits, e.g., the definitions of
clients, jobs and filesets. The basic configuration (database
settings, logging etc.) must be managed in a handwritten
bareos-dir.conf. This file should include the configuration
generated by Puppet using the following snippet:
# Puppet clients and jobs
@|"bash -c 'ls -d /etc/bareos/{clients,jobs,filesets}.d/*.conf | sed s/^/@/'"Some pieces of the configuration must be written by hand or handled otherwise. This includes schedules and jobdefs.
In order to get a working setup, a few common Hiera keys must be set:
bareos::secret: A random string which is hashed with a seed
value for each client to create its password.
bareos::director: The name of the director. Not necessarily a
hostname. It is essentially the director's username when
authenticating with a FD. It is also used in the tag of exported
resources. Default: "dump-dir"
bareos::schedules: A hash containing sets of schedules. Each
key defines a set, the value for that key is an array of schedule
names. A declared job will look for the set called normal by
default. The schedules themselves must be defined outside this module.
__bareos::default_jobdef__: Name of default job definition.
Defaults to DefaultJob. The jobdef itself must be
defined outside this module.
__bareos::security_zone__: A tag identifying which secret the
director should use to generate the password for this client.
When security zone is unset, bareos::secret will be used on both
client and server. If security zone is set, the client will still use
the bareos::secret as seen in its view of Hiera, but the server will
instead look for the alternate secret in bareos::server::secrets.
The bareos::server class installs the software and collects exported
resources associated with it.
bareos::server::secrets: Hash containing secrets for other
security zones. Default: {}
bareos::server::client_file_prefix: Where to put collected
client resources. Default: '/etc/bareos/clients.d/'
bareos::server::job_file_prefix: Where to put collected job
resources. Default: '/etc/bareos/jobs.d/'
bareos::server::fileset_file_prefix: Where to put collected
fileset resources. Default: '/etc/bareos/filesets.d/'
The values bareos::schedules, bareos::secret, bareos::server::secrets
are taken directly from Hiera. Users without Hiera can pass the parameters
$default_schedules and $default_secret to class bareos and the
parameter $default_secrets to class bareos::server.
The bareos::client class will install the file daemon and configure
it with passwords. It will also export resources (Client, Job, and
Fileset) which the director collects.
bareos::client::password:
Set this parameter to get the same password on several clients. This
is not the actual password used in the configuration files, this is
just a seed which is hashed with ${bareos::secret}. We do that
extra step to avoid putting the actual password in PuppetDB. Default: FQDN
bareos::client::concurrency:
How many jobs the director will schedule at the same time on this
client. The FD will be configured to allow 10 more connections than
this. Default: 10
bareos::client::implementation:
Either bacula or bareos. This needs to be set via Hiera to affect
defaults which are based on this value. Default: "bacula"
__bareos::client::client_name__:
The name of the client, without the "-fd" suffix. Default: FQDN
__bareos::client::name_suffix__:
The suffix to use. Default: "-fd"
bareos::client::address:
The address or hostname the director should connect to. Default: FQDN
bareos::client::passive:
Turn on passive mode, which makes SD connect to FD rather than the
other way round. Requires Bareos 14 and above. Default: false
bareos::client::client_initiated_connection:
Turn on Client Initiated Connection, which makes FD connect to DIR on
startup and leave the connection up. This enables the FD to have a
variable IP address. Requires Bareos 16 and above. Default: false
__bareos::client::job_retention__:
How long to keep jobs from this client in the database. Default: "180d"
__bareos::client::file_retention__:
How long to keep detailed information about backup job contents in the
database. Default: "60d"
bareos::client::monitors:
Additional list of monitors to add to bacula-fd.conf. Typical use:
bareos::client::monitors:
tray-mon:
password: password-in-plain-textUse eyaml to protect "password-in-plain-text". All keys in the hash are added as parameters to the Director directive.
bareos::client::compression: This variable is only used as a
default for filesets declared on this host.
bareos::client::fstype: This variable is only used as a
default for filesets declared on this host.
__bareos::client::exclude_paths: This variable is only__ used as
a default for filesets declared on this host.
__bareos::client::exclude_patterns: This variable is only__ used as
a default for filesets declared on this host.
__bareos::client::backup_dir__: The default parent directory where
the preset jobs will dump data. Default: "/var/backups"
bareos::client::manage_backup_dir: Create and manage the default
parent directory. Default: true
bareos::client::backup_dir_owner: Owner of above directory
Default: "root"
bareos::client::backup_dir_group: Group of above directory
Default: "root"
bareos::client::backup_dir_mode: Mode of above directory
Default: "0755"
__bareos::client::systemd_limits__: Hash of resource limits which
needs overriding. Only works for systemd, but no check is done to see
if systemd manages the service. Example: { 'nofile' => 16384 }.
Default is to do nothing.
bareos::client::ipv6: Whether to listen on IPv6 when available.
This is by default set to false on Windows due to a bug in Bareos.
__bareos::client::tls_enable__: Whether to enable TLS. Set to true
or false, default is to omit directive. (Proper TLS support will come
in a later version of this module.)
bareos::client::package: The name of the package(s) to install.
The default is chosen based on operatingsystem and implementation. On
Windows this is set to an empty array since the package name contains
the version number, and Puppet usually will not be able to install the
package. In other words, the Bareos client must be installed before
Puppet is run.
In addition, you can manage the service name, the port number, the location of the log file, the pid file and the working directory, but this should not be necessary to do.
Jobs are defined in the bareos::client::jobs hash.
By default, a "system" job will be enabled for the client, using the
default jobdef. If you define a "system" fileset in the
bareos::client::filesets hash, it will be used with this default job.
(It will not work for filesets declared directly with the
bareos::client::fileset define.)
Each key in the hash becomes the name of the resource. This is added
to the client name and used as the name of the job. A
bareos::job_definition with that name will be exported for the
director to pick up.
__job_name__: Specify the full job name explicitly.
jobdef: The name of the job defaults. Default: $bareos::default_jobdef
fileset: The name of the fileset to use. When set, overrides
the fileset defined in the jobdef. This can be the full name of the
fileset, but also the abbreviated name used in
bareos::client::filesets. If unset, it will look for short name of
this job (e.g., "system"), and use that if it exists.
__schedule_set__: The name of the list of schedules to pick randomly
from. Default: normal
sched: Explicit name of schedule, overrides random selection.
(schedule is a reserved word in Puppet, hence the strange parameter name.)
accurate: Whether to turn Accurate on or off. When set to '',
don't include directive in configuration (job defaults are used).
Default: ''
order: Give a hint to in what order jobs should run. This
should be a value consisting of one capital letter and two digits.
Jobs with "A00" will be scheduled first, "Z99" will be scheduled last.
The default is "N50".
runscript: Array of script specifications to run before or after
job. Each element is a hash containing values for command and
optional other parameters. command can be a single command or an
array of strings. runswhen is either before or after, by
default it is before. Other parameters are written verbatim as Key = value to bareos configuration.
preset: Use specified define to export the job. See next
section. If unqualified, bareos::job::preset:: will be prepended.
__preset_params__: Parameters to pass to preset class.
base_job_name: Optional name of a Base backup job. If Bareos
has a valid copy of this Base job, a Full backup will be relative to
that base.
make_base_job: Boolean which controls whether a companion Base
level job should be defined. The base backup job definition will be
the same as this job, but with a different schedule and jobdefs.
Default is false.
__base_schedule__: What schedule set to use for the Base job. Must
have the same number of entries as the schedule set for the job
itself. Default is 'base'.
__base_jobdef__: The jobdefs to use for the Base job. Default is
'BaseJob'.
The following example installs a backup agent and registers a job with all the default settings:
include bareos::clientTo only install the client without running any jobs, add to Hiera:
bareos::client::jobs: {}This example also runs a normal full backup, but later than normal
(this assumes that a set called late is defined in
bareos::schedules):
bareos::client::jobs:
system:
schedule_set: lateA preset define can install scripts or other software on the client in addition to exporting configuration for the backup server.
This preset installs the script mysqldumpbackup and installs a configuration file for it. See code for full list of parameters.
Example usage:
bareos::client::jobs:
system:
preset: bareos::job::preset::mysqldumpbackup
preset_params:
keep_backup: 5
backup_dir: /srv/mysql/backupIf using MariaDB, add this to the preset_params block:
servicename: mariadbThis preset installs the package mylvmbackup and installs a configuration file for it. See code for full list of parameters.
Example usage:
bareos::client::jobs:
system:
preset: bareos::job::preset::mylvmbackup
preset_params:
keep_backup: 5
vgname: sysvg
lvname: mysqlThis preset installs the package percona-xtrabackup and the percona plugin from bareos-contrib. (Since the plugin is not packaged, we distribute a copy of it in this module.) The backup will include a "virtual" file in xbstream format and the binary logs. You should still include a normal system backup.
Available settings in preset_params include:
mycnf: location of my.cnf to use
__skip_binlog__: do not include binlogs in backup, default is false
__xtrabackup_package__: name of package containing xtrabackup(1).
On Ubuntu Xenial you may need to specify "percona-xtrabackup-24".
Default: "percona-xtrabackup"
Example usage:
bareos::client::implementation: bareos
bareos::client::jobs:
system: {}
db:
preset: bareos::job::preset::percona
preset_params:
mycnf: /etc/mysql/mysql-db02.cnfThis preset installs the script pgdumpbackup and installs a configuration
file for it. It has the following params, passed via preset_params:
instance: name of instance. Needed if there is more than one
job using this preset. Will be added as argument to pgdumpbackup
unless it is default.
ignore_not_running: if true, exit silently without taking backup
if postgresql is not running. Default is false.
__keep_backup__: how many days to keep backup
__backup_dir__: where to store backups
backup_dir_owner: what user should own backup_dir. Default: 'postgres'
backup_dir_group: what group should own backup_dir. Default: 'root'
backup_dir_mode: what permissions to use on backup_dir. Default: '0750'
server: server name to connect to (default is local socket)
initscript: to check if service is running
cluster: what cluster to dump (default "", which means connect to port 5432)
__skip_databases__: array of databases to skip
__log_method__: where to log. default is "console" (ie., stderr)
__syslog_facility__: where to log. default is 'daemon'
environ: array of extra environment variables (example: ["HOME=/root"])
Example usage:
bareos::client::jobs:
system:
preset: bareos::job::preset::pgdumpbackup
preset_params:
cluster: 9.2/mainThis preset installs an S3 plugin for Bareos (Bacula is not supported).
(Since the plugin is not packaged, we distribute a copy of it in this
module.) The backup will include a "virtual" file directory tree
/situla with a traversed copy of the bucket inside. A script which
calls radosgw-admin to create an authentication file is also included.
Available settings in preset_params include:
__user_name__: who to authenticate as
bucket: what bucket to backup.
prefix: an optional prefix to restrict the backup
pattern: an optional regexp to restrict the backup to matching
files.
For convenience, the job name can be used to specify these values, if
it is on the format anything:user_name:bucket:anything. bucket is
optional, if it is unspecified, it is assumed to be the same as the
user name.
Example usage:
bareos::client::implementation: bareos
bareos::client::jobs:
# backup of bucket websrv as user websrv
"s3:websrv":
preset: bareos::job::preset::s3
preset_params:
pattern: "\.pdf$"
# backup of bucket aux as user websrv
"s3:websrv:aux":
preset: bareos::job::preset::s3
# backup of bucket aux2, subdirectory img, as user websrv
"s3:websrv:aux2:img":
preset: bareos::job::preset::s3
preset_params:
prefix: img
# backup of bucket aux2, subdirectory docs, as user websrv
"docs":
preset: bareos::job::preset::s3
preset_params:
user_name: websrv
bucket: aux2
prefix: docsThe signature for a preset should be this:
define widget::backup::preset::widgetdump(
$short_title,
$client_name,
$jobdef,
$fileset,
$sched,
$order,
$runscript,
$params,
)title for the define will be the full job name.
short_title will be the short job name, for your convenience.
client_name is the name of the client, and should be passed on.
jobdef will be the empty string if the user didn't specify a jobdef
explicitly. You should respect the explicit value, but replace the
empty string with a value which works for your preset (possibly
$bareos::default_jobdef). New job defaults can not be defined in
Puppet code, they must be added to the main Bareos configuration
manually.
fileset will normally be empty, and should just be passed on.
sched is the chosen schedule for this job, pass it on.
order should just be passed on.
params is the hash passed by the user as preset_params. The
preset is free to specify its format and content.
runscript is an array of additional pre- or post commands.
A normal job exports a bareos::job_define which bareos::server
picks up. When a preset is used, exporting that declaration must be
done by its define.
This should be done like this:
$_jobdef = $jobdef ? { '' => 'WidgetJob', default => $jobdef }
@@bareos::job_definition {
$title:
client_name => $client_name,
name_suffix => $bareos::client::name_suffix,
base => $base,
jobdef => $_jobdef,
fileset => $fileset,
sched => $sched,
order => $order,
runscript => flatten([ $runscript,
[{ 'command' => '/usr/local/bin/widgetdump' }]
]),
tag => "bareos::server::${bareos::director}"
}Almost all of the above code must be copied more or less verbatim.
flatten() is used to concatenate arrays, which is otherwise only
available when the future parser is enabled. If you don't need a
runscript, you just pass $runscript on.
You should try to write your define so it can be used more than once
per client, i.e., consider using ensure_resource('file', { ... })
instead of just file to avoid duplicate declarations.
The support for filesets is not complete, it is kept simple to focus on filesets with static include and exclude rules.
Since the normal filesets should contain Exclude dir containing
.nobackup, an alternative to making a custom fileset may be to
manage strategically placed .nobackup files using Puppet.
The name (title) of the bareos::client::fileset instance is added to
the client name and used as the name of the fileset in the Bareos
configuration. The define exports a fileset_definition which the
director will pick up.
__fileset_name__: Specify the fileset name explicitly.
__client_name__: Used as the first part of a fully qualified fileset
name. Doesn't need to relate to any host or client names as long as
the result is unique. Default is bareos::client::client_name.
__include_paths__: Array of paths to include in backup. Mandatory,
no default.
__exclude_paths__: Array of paths to exclude. If unset, use the
client's default (bareos::client::exclude_paths). If one of the
values is "defaults", the default exclude list will be added to the
array.
__exclude_patterns__: Hash of different kinds of patterns to
exclude. If unset, use the client's default
(bareos::client::exclude_patterns). The possible keys are wild_dir,
wild_file, regex_dir and regex_file. Each key in the hash can
have a list of patterns as its value.
exclude_dir_containing: Directories containing a file with this
name will be skipped. Set to "" to disable functionality. Default:
".nobackup".
__include_patterns__: Hash of different kinds of patterns to
include. Default is empty (no special rules). The possible keys are
wild_dir, wild_file, regex_dir and regex_file. Each key in
the hash can have a list of patterns as its value. In Bareos, all
files will be included by default, so specifying an extra include rule
changes nothing. Therefore, an exclude rule matching all files will
be added to the configuration as well. It is recommended to manually
inspect the result on the server to see if it works as intended. This
is especially important for the wild_dir and regex_dir directives.
__ignore_changes__: If fileset changes, rerun Full backup if this is
set to false. Default: true
__acl_support__: Include information about ACLs in backup. Causes
an extra system call per file. Default: true
compression: What compression algorithm to use. To disable
compression, set to false. Default: "GZIP"
sparse: Whether to store information about the holes in sparse
files or just store the holes as plain NUL bytes. Default: true
onefs: Whether to recurse into mount points. Default:
false (do not recurse).
fstype: If onefs is false (the default), this array lists the
filesystem types which should be recursed into (and backed up). The
default (from bareos::client::fstype) contains the normal local
filesystems, like ext4 and xfs, but not nfs or cifs.
bareos::client::filesets:
only_srv:
include_paths:
- /srv
exclude_paths:
- /srv/cacheIf this configuration is used on node foo.example.com, a fileset
called "foo.example.com-only_srv" will be exported.
The following will install the software and register two jobs, "system" and "srv". Before the "srv" job runs, the prepare script will run, and if it fails, backup will be aborted. After the "srv" job finishes, the cleanup script will run.
bareos::client::jobs:
system:
fileset: "system"
srv:
fileset: "srv"
runscript:
- command: "/usr/local/sbin/prepare"
abortjobonerror: true
- command: "/usr/local/sbin/cleanup"
runswhen: after
bareos::client::filesets:
system:
include_paths:
- /
exclude_paths:
- /srv
srv:
include_paths:
- /srvThe default fileset will not traverse into NFS file systems, so it
needs to be specified explicitly. Here we define a separate job for
NFS paths. We set OneFS to true, since the default fstype list does
not include nfs. Alternatively, if we want to recurse into other NFS
filesystems, we could set fstype: ["nfs"]
bareos::client::filesets:
nfs:
onefs: true
include_paths:
- /srv/dataWe add another job which uses the new fileset (the system job still
uses the default fileset specified in the $bareos::default_jobdef)
bareos::client::jobs:
system: {}
nfs:
fileset: "nfs"When doing backup of a service which can run on more than one node, it
is essential to set bareos::client::password to the same value on
each of the nodes. It is also important that the job and fileset
definitions agree across nodes, so it is best to put the configuration
in a single Hiera file read by the relevant nodes.
Example:
bareos::client::password: common-secret
bareos::client::jobs:
system:
fileset: without_archive
archive_service:
client_name: archive.example.com
fileset_name: archive
bareos::client::filesets:
without_archive:
client_name: archive.example.com
include_paths:
- /
exclude_paths:
- /srv/archive
archive:
client_name: archive.example.com
include_paths:
- /srv/archive
bareos::client::service_addr:
archive.example.com: {}This will make one job "${fqdn}-system-job" for each node using this configuration, and one job "archive.example.com-archive_service-job". The latter job will run on the extra client, "archive.example.com". (The name must resolve in DNS, otherwise specify "address".)
It will declare one fileset, "archive.example.com-without_archive", which all the "${fqdn}-system-job" jobs use. If the fileset didn't specify "client_name", each node would declare its own copy of the fileset called "${fqdn}-without_archive". This would work fine, but it is always good to avoid duplication.
Finally, it will declare one fileset "archive.example.com-archive", which the "archive.example.com-archive_service-job" uses.