tiredofit / docker-db-backup

Backup multiple database types on a scheduled basis with many customizable options
MIT License
846 stars 124 forks source link

MODE=MANUAL and MANUAL_RUN_FOREVER=FALSE not working in 4.0.19 #290

Closed ruarkadi closed 10 months ago

ruarkadi commented 10 months ago

Summary

I am wrapping the docker-db-backup in a kubernetes cronjob yaml manifest with all the (supposedly) correct environment variables indicated in the README.md file for these cases. My goal is to have the system run its' jobs and exit.

However, it finished the very small backup and then just loops around to sleeping instead of existing. It appears as though the "persist" param in install/assets/dbbackup/template-dbbackup/run isn't set properly. Perhaps at this section: https://github.com/tiredofit/docker-db-backup/blob/674a98fcd8b0abe4fa4b08888255097885250171/install/assets/dbbackup/template-dbbackup/run#L19

Steps to reproduce

Create a CronJob with similar specs:

apiVersion: batch/v1
kind: CronJob
metadata:
  labels:
    app.kubernetes.io/component: mongodump
    app.kubernetes.io/instance: mongodb-gmfc-rkd
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: mongodb
    app.kubernetes.io/version: 7.0.3
    argocd.argoproj.io/instance: mongodb-gmfc-rkd
    helm.sh/chart: mongodb-14.3.0
  name: mongodb-mongodump
  namespace: mongodb
  resourceVersion: '7331833'
  uid: c10456d1-6646-4302-9e27-1a543c1b4ef5
spec:
  concurrencyPolicy: Replace
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            app.kubernetes.io/component: mongodump
            app.kubernetes.io/instance: mongodb-gmfc-rkd
            app.kubernetes.io/managed-by: Helm
            app.kubernetes.io/name: mongodb
            app.kubernetes.io/version: 7.0.3
            helm.sh/chart: mongodb-14.3.0
        spec:
          containers:
            - env:
                - name: MODE
                  value: MANUAL
                - name: MANUAL_RUN_FOREVER
                  value: 'FALSE'
                - name: BACKUP_JOB_CONCURRENCY
                  value: '1'
                - name: CONTAINER_ENABLE_MONITORING
                  value: 'FALSE'
                - name: CONTAINER_ENABLE_SCHEDULING
                  value: 'FALSE'
                - name: CONTAINER_ENABLE_MESSAGING
                  value: 'FALSE'
                - name: CONTAINER_ENABLE_LOGROTATE
                  value: 'FALSE'
                - name: TIMEZONE
                  value: Etc/UTC
                - name: CONTAINER_NAME
                  value: mongodb-mongodump
                - name: DEFAULT_LOG_LEVEL
                  value: DEBUG
                - name: DEBUG_MODE
                  value: 'TRUE'
                - name: DEFAULT_COMPRESSION
                  value: GZ
                - name: DEFAULT_BACKUP_LOCATION
                  value: S3
                - name: DEFAULT_S3_BUCKET
                  value: XXXXXXXXXX
                - name: DEFAULT_S3_KEY_ID
                  value: >-
                    XXXXXXXXXXXXXXXXX
                - name: DEFAULT_S3_KEY_SECRET
                  value: XXXXXXXXXXXX
                - name: DEFAULT_S3_PATH
                  value: TEST/2023/11/21
                - name: DEFAULT_S3_REGION
                  value: us
                - name: DEFAULT_S3_HOST
                  value: storage.googleapis.com
                - name: DB01_TYPE
                  value: mongo
                - name: DB01_HOST
                  value: mongodb-headless
                - name: DB01_NAME
                  value: base_monitor
                - name: DB01_AUTH
                  value: admin
                - name: DB01_USER
                  value: test
                - name: DB01_PASS
                  valueFrom:
                    secretKeyRef:
                      key: mongodb-root-password
                      name: mongodb-secret-mongodb
              image: 'tiredofit/db-backup:4.0.19'
              imagePullPolicy: IfNotPresent
              name: mongodb-mongodump
              resources: {}
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
          dnsPolicy: ClusterFirst
          restartPolicy: OnFailure
          schedulerName: default-scheduler
          securityContext: {}
          terminationGracePeriodSeconds: 30
  schedule: '*/30 * * * *'
  successfulJobsHistoryLimit: 3
  suspend: false

The manifest above is templated from the following helm template as an ExtraDeploy in bitnami mongodb chart:

            apiVersion: batch/v1
            kind: CronJob
            metadata:
              name: {{ include "mongodb.fullname" . }}-mongodump
              namespace: {{ include "mongodb.namespace" . | quote }}
              labels: {{- include "common.labels.standard" . | nindent 4 }}
                app.kubernetes.io/component: mongodump
                {{- if .Values.commonLabels }}
                {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
                {{- end }}
              {{- if .Values.commonAnnotations }}
              annotations:
                {{- if .Values.commonAnnotations }}
                {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
                {{- end }}
              {{- end }}
            spec:
              concurrencyPolicy: Replace
              schedule: '*/30 * * * *'
              jobTemplate:
                spec:
                  template:
                    metadata:
                      labels: {{- include "common.labels.standard" . | nindent 12 }}
                        app.kubernetes.io/component: mongodump
                        {{- if .Values.commonLabels }}
                        {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 12 }}
                        {{- end }}
                      {{- if or .Values.annotations .Values.commonAnnotations }}
                      annotations:
                        {{- if .Values.commonAnnotations }}
                        {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 12 }}
                        {{- end }}
                      {{- end }}
                    spec:
                      restartPolicy: OnFailure
                      containers:
                      - name: {{ include "mongodb.fullname" . }}-mongodump
                        image: "tiredofit/db-backup:4.0.19"
                        env:
                        - name: MODE
                          value: "MANUAL"
                        - name: MANUAL_RUN_FOREVER
                          value: "FALSE"
                        - name: BACKUP_JOB_CONCURRENCY
                          value: "1"
                        - name: CONTAINER_ENABLE_MONITORING
                          value: "FALSE"
                        - name: CONTAINER_ENABLE_SCHEDULING
                          value: "FALSE"
                        - name: CONTAINER_ENABLE_MESSAGING
                          value: "FALSE"
                        - name: CONTAINER_ENABLE_LOGROTATE
                          value: "FALSE"
                        - name: TIMEZONE
                          value: Etc/UTC
                        - name: CONTAINER_NAME
                          value: "{{ include "mongodb.fullname" . }}-mongodump"
                        - name: DEFAULT_LOG_LEVEL
                          value: DEBUG
                        - name: DEBUG_MODE
                          value: "TRUE"                        
                        - name: DEFAULT_COMPRESSION
                          value: GZ
                        - name: DEFAULT_BACKUP_LOCATION
                          value: S3
                        - name: DEFAULT_S3_BUCKET
                          value: test-bucket
                        - name: DEFAULT_S3_KEY_ID
                          value: "key"
                        - name: DEFAULT_S3_KEY_SECRET
                          value: "secret"
                        - name: DEFAULT_S3_PATH
                          value: {{ .Values.global.clusterName | upper }}/{{ now | date "2006/01/02" }}
                        - name: DEFAULT_S3_REGION
                          value: us
                        - name: DEFAULT_S3_HOST
                          value: storage.googleapis.com
                        - name: DB01_TYPE
                          value: mongo
                        - name: DB01_HOST
                          value: {{ include "mongodb.service.nameOverride" . }}
                        - name: DB01_NAME
                          value: test
                        - name: DB01_AUTH
                          value: admin
                        - name: DB01_USER
                          value: {{ .Values.auth.rootUser | quote }}
                        - name: DB01_PASS
                          valueFrom:
                            secretKeyRef:
                              name: {{ include "mongodb.secretName" . }}
                              key: mongodb-root-password

What is the expected correct behavior?

Have this bit of code working correctly:

    symlink_log
    cleanup_old_data

    if var_false "${persist}" ; then
        print_debug "Exiting due to manual mode"
        exit "${exitcode_backup}";

Relevant logs and/or screenshots

Debug log below

env command output from within the container:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=mongodb-mongodump-28342770-xtphc
FLUENTBIT_VERSION=2.2.0
S6_OVERLAY_VERSION=3.1.6.2
YQ_VERSION=v4.35.2
ZABBIX_VERSION=6.4.8
DOAS_VERSION=v6.8.2
DEBUG_MODE=TRUE
TIMEZONE=Etc/UTC
CONTAINER_ENABLE_SCHEDULING=FALSE
CONTAINER_SCHEDULING_BACKEND=cron
CONTAINER_ENABLE_MESSAGING=FALSE
CONTAINER_MESSAGING_BACKEND=msmtp
CONTAINER_ENABLE_MONITORING=FALSE
CONTAINER_MONITORING_BACKEND=zabbix
CONTAINER_ENABLE_LOGSHIPPING=FALSE
S6_GLOBAL_PATH=/command:/usr/bin:/bin:/usr/sbin:sbin:/usr/local/bin:/usr/local/sbin
S6_KEEP_ENV=1
S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
IMAGE_NAME=tiredofit/db-backup
IMAGE_REPO_URL=https://github.com/tiredofit/docker-db-backup/
INFLUX1_CLIENT_VERSION=1.8.0
INFLUX2_CLIENT_VERSION=2.7.3
MSODBC_VERSION=18.3.2.1-1
MSSQL_VERSION=18.3.1.1-1
AWS_CLI_VERSION=1.29.78
DEFAULT_S3_KEY_SECRET=KEY_SECRET
DEFAULT_S3_PATH=TEST/2023/11/21
DEFAULT_S3_REGION=us
MODE=MANUAL
BACKUP_JOB_CONCURRENCY=1
DEFAULT_BACKUP_LOCATION=S3
DEFAULT_S3_BUCKET=testbucket
DB01_TYPE=mongo
DB01_HOST=mongodb-headless
DB01_USER=csr
MANUAL_RUN_FOREVER=FALSE
CONTAINER_ENABLE_LOGROTATE=FALSE
DEFAULT_LOG_LEVEL=DEBUG
DEFAULT_S3_KEY_ID=KEY_ID
DEFAULT_S3_HOST=storage.googleapis.com
CONTAINER_NAME=mongodb-mongodump
DEFAULT_COMPRESSION=GZ
DB01_PASS=dummypass
DB01_NAME=base_monitor
DB01_AUTH=admin
MONGO_EXPRESS_GMFC_RKD_PORT=tcp://10.43.74.157:8081
MONGO_EXPRESS_GMFC_RKD_PORT_8081_TCP_PROTO=tcp
MONGO_EXPRESS_GMFC_RKD_PORT_8081_TCP_ADDR=10.43.74.157
MONGODB_METRICS_SERVICE_HOST=10.43.215.128
MONGODB_METRICS_SERVICE_PORT=9216
MONGODB_METRICS_PORT_9216_TCP=tcp://10.43.215.128:9216
KUBERNETES_SERVICE_HOST=10.43.0.1
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1
MONGO_EXPRESS_GMFC_RKD_PORT_8081_TCP_PORT=8081
MONGODB_METRICS_PORT_9216_TCP_PROTO=tcp
MONGODB_METRICS_PORT_9216_TCP_ADDR=10.43.215.128
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
MONGO_EXPRESS_GMFC_RKD_SERVICE_PORT_HTTP=8081
MONGODB_METRICS_SERVICE_PORT_HTTP_METRICS=9216
KUBERNETES_PORT_443_TCP=tcp://10.43.0.1:443
KUBERNETES_PORT=tcp://10.43.0.1:443
MONGO_EXPRESS_GMFC_RKD_SERVICE_HOST=10.43.74.157
MONGO_EXPRESS_GMFC_RKD_SERVICE_PORT=8081
MONGO_EXPRESS_GMFC_RKD_PORT_8081_TCP=tcp://10.43.74.157:8081
MONGODB_METRICS_PORT=tcp://10.43.215.128:9216
MONGODB_METRICS_PORT_9216_TCP_PORT=9216
KUBERNETES_SERVICE_PORT_HTTPS=443
TERM=xterm
HOME=/root

Environment

This is a K3S cluster running version: v1.26.6+k3s1 I have added this CronJob as part of the bitnami mongodb chart version: mongodb-14.3.0 It is deployed using ArgoCD version: v2.8.3+77556d9

Any logs | docker-compose.yml

Possible fixes

ruarkadi commented 10 months ago
+ output_off
+ '[' true = true ']'
+ set +x
                                                                       ,---.
,--------.,--.                 ,--.            ,---.    ,--. ,--------.|   |
'--.  .--'`--',--.--. ,---.  ,-|  |     ,---. /  .-'    |  | '--.  .--'|  .'
   |  |   ,--.|  .--'| .-. :' .-. |    | .-. ||  `-,    |  |    |  |   |  |
   |  |   |  ||  |   \   --.\ `-' |    ' '-' '|  .-'    |  |.--.|  |   `--'
   `--'   `--'`--'    `----' `---'      `---' `--'      `--''--'`--'   .--.
                                                                       '--'
Image:  tiredofit/db-backup | Version  4.0.19 Type 'image_changelog' for details
Repository/Documentation: https://github.com/tiredofit/docker-db-backup/

If this image provides you value  - Consider sponsoring my work for continued
development, timely updates, and feature requests. Commercial support available.

                    More Info:  https://www.tiredofit.ca

2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/00-startup ** [container-init] Creating Logrotate Entry for /var/log/container//container.log
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/00-startup ** [container-init] Cold Container startup detected - Proceeding to initialize services normally
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/00-startup ** [container-init] Adding shortcuts to .bashrc
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/00-startup ** [container-init] Adding 'changelog' command to .bashrc
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:01 [NOTICE] /etc/cont-init.d/01-timezone ** [timezone] Timezone: Setting to 'Etc/UTC' from 'Etc/GMT'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/02-permissions ** [permissions] Enabling permissions features
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/03-monitoring ** [container] Container: Getting defaults for '03-monitoring'
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/03-monitoring ** [container] Container: No functions available for '03-monitoring'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/04-scheduling ** [container] Container: Getting defaults for '04-scheduling'
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/04-scheduling ** [container] Container: No functions available for '04-scheduling'
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/04-scheduling ** [scheduling] Disabling Scheduling
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/00-container
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/02-permissions
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/03-monitoring
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/04-scheduling
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/05-logging
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/06-messaging
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/07-firewall
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting defaults for /assets/defaults/10-db-backup
2023-11-21.11:30:01 [DEBUG] /etc/cont-init.d/05-logging ** [container] Container: Getting functions for /assets/functions/10-db-backup
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/06-messaging ** [container] Container: Getting defaults for '06-messaging'
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/06-messaging ** [container] Container: No functions available for '06-messaging'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/00-container
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/02-permissions
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/03-monitoring
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/04-scheduling
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/05-logging
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/06-messaging
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/07-firewall
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting defaults for /assets/defaults/10-db-backup
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/07-firewall ** [container] Container: Getting functions for /assets/functions/10-db-backup
+ prepare_service single
+ '[' -n single ']'
+ case "$(dirname "$0")" in
++ dirname /etc/cont-init.d/10-db-backup
+ set -a
+ get_defaults single
+ '[' -n single ']'
+ case "${1,,}" in
+ case "$(dirname "$0")" in
++ dirname /etc/cont-init.d/10-db-backup
++ basename /etc/cont-init.d/10-db-backup
+ '[' -f /assets/defaults/10-db-backup ']'
++ basename /etc/cont-init.d/10-db-backup
+ print_debug 'Container: Getting defaults for '\''10-db-backup'\'''
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/10-db-backup ** [container] Container: Getting defaults for '10-db-backup'
+ case "$(basename "$0")" in
++ basename /etc/cont-init.d/10-db-backup
++ basename /etc/cont-init.d/10-db-backup
+ source /assets/defaults/10-db-backup
++ BACKUP_JOB_CONCURRENCY=1
++ DBBACKUP_USER=dbbackup
++ DBBACKUP_GROUP=dbbackup
++ DEFAULT_BACKUP_BEGIN=+0
++ DEFAULT_BACKUP_INTERVAL=1440
++ DEFAULT_BACKUP_INTERVAL=1440
++ DEFAULT_BACKUP_LOCATION=S3
++ DEFAULT_BLOBXFER_REMOTE_PATH=/docker-db-backup
++ DEFAULT_CHECKSUM=MD5
++ DEFAULT_COMPRESSION=GZ
++ DEFAULT_COMPRESSION_LEVEL=3
++ DEFAULT_CREATE_LATEST_SYMLINK=TRUE
++ DEFAULT_ENABLE_PARALLEL_COMPRESSION=TRUE
++ DEFAULT_ENCRYPT=FALSE
++ DEFAULT_FILESYSTEM_PATH=/backup
++ DEFAULT_FILESYSTEM_PATH_PERMISSION=700
++ DEFAULT_FILESYSTEM_PERMISSION=600
++ DEFAULT_FILESYSTEM_ARCHIVE_PATH=/backup/archive/
++ DEFAULT_LOG_LEVEL=DEBUG
++ DEFAULT_MYSQL_ENABLE_TLS=FALSE
++ DEFAULT_MYSQL_EVENTS=TRUE
++ DEFAULT_MYSQL_MAX_ALLOWED_PACKET=512M
++ DEFAULT_MYSQL_SINGLE_TRANSACTION=TRUE
++ DEFAULT_MYSQL_STORED_PROCEDURES=TRUE
++ DEFAULT_MYSQL_TLS_CA_FILE=/etc/ssl/cert.pem
++ DEFAULT_MYSQL_TLS_VERIFY=FALSE
++ DEFAULT_MYSQL_TLS_VERSION=TLSv1.1,TLSv1.2,TLSv1.3
++ DEFAULT_MSSQL_MODE=database
+++ nproc
++ DEFAULT_PARALLEL_COMPRESSION_THREADS=4
++ DEFAULT_RESOURCE_OPTIMIZED=FALSE
++ DEFAULT_S3_CERT_SKIP_VERIFY=TRUE
++ DEFAULT_S3_PROTOCOL=https
++ DEFAULT_SCRIPT_LOCATION_PRE=/assets/scripts/pre/
++ DEFAULT_SCRIPT_LOCATION_POST=/assets/scripts/post/
++ DEFAULT_SIZE_VALUE=bytes
++ DEFAULT_SKIP_AVAILABILITY_CHECK=FALSE
++ DEFAULT_SPLIT_DB=TRUE
++ LOG_PATH=/logs
++ MANUAL_RUN_FOREVER=FALSE
++ MODE=MANUAL
++ TEMP_PATH=/tmp/backups
++ '[' -n '' ']'
+ get_functions single
+ '[' -n single ']'
+ case "${1,,}" in
+ case "$(dirname "$0")" in
++ dirname /etc/cont-init.d/10-db-backup
++ basename /etc/cont-init.d/10-db-backup
+ '[' -f /assets/functions/10-db-backup ']'
++ basename /etc/cont-init.d/10-db-backup
+ print_debug 'Container: Getting functions for '\''10-db-backup'\'''
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/10-db-backup ** [container] Container: Getting functions for '10-db-backup'
++ basename /etc/cont-init.d/10-db-backup
+ source /assets/functions/10-db-backup
+ set +a
+ prepare_service 03-monitoring
+ '[' -n 03-monitoring ']'
+ case "$(dirname "$0")" in
++ dirname /etc/cont-init.d/10-db-backup
+ set -a
+ get_defaults 03-monitoring
+ '[' -n 03-monitoring ']'
+ case "${1,,}" in
++ echo 03-monitoring
++ tr , '\n'
+ defaults=03-monitoring
+ for d in $defaults
+ '[' -f /assets/defaults/03-monitoring ']'
+ print_debug 'Container: Getting defaults for 03-monitoring'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/10-db-backup ** [container] Container: Getting defaults for 03-monitoring
+ case "${d}" in
+ output_off
+ '[' true = true ']'
+ set +x
+ get_functions 03-monitoring
+ '[' -n 03-monitoring ']'
+ case "${1,,}" in
++ echo 03-monitoring
++ tr , '\n'
+ functions=03-monitoring
+ for f in $functions
+ '[' -f /assets/functions/03-monitoring ']'
+ print_debug 'Container: No functions available for '\''03-monitoring'\'''
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/10-db-backup ** [container] Container: No functions available for '03-monitoring'
+ set +a
+ PROCESS_NAME=db-backup
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [INFO] /etc/cont-init.d/10-db-backup ** [db-backup] Running in Manual mode - Execute 'backup_now' to perform a manual backup
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename /etc/cont-init.d/10-db-backup
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ service_stop 10-db-backup
++ basename /etc/cont-init.d/10-db-backup
+ '[' 10-db-backup '!=' 10-db-backup ']'
+ DONOTSTART=TRUE
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ db_backup_container_init
+ rm -rf /tmp/.container/db-backup-backups
+ echo '0 0 * * * /usr/sbin/logrotate_dbbackup >/dev/null 2>&1'
/assets/functions/10-db-backup: line 1205: /assets/cron/dbbackup_logrotate: No such file or directory
+ touch /tmp/.container/db-backup-backups
+ create_schedulers backup
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ backup
++ set -o posix
++ set
++ grep -Pc '^(DB[0-9]._HOST=|.*MONGO_CUSTOM_URI=)'
+ local backup_instances=1
+ print_debug '[create_schedulers] Found '\''1'\'' DB_HOST instances'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/10-db-backup ** [db-backup] [create_schedulers] Found '1' DB_HOST instances
+ '[' -n '' ']'
+ (( instance = 01 ))
+ (( instance <= backup_instances ))
++ printf %02d 1
+ instance=01
+ cp -R /assets/dbbackup/template-dbbackup /etc/services.available/dbbackup-01
+ sed -i 's|{{BACKUP_NUMBER}}|01|g' /etc/services.available/dbbackup-01/run
+ cat
+ chmod +x /usr/bin/backup01-now
+ '[' 01 = 01 ']'
+ touch /usr/bin/backup-now
+ chmod +x /usr/bin/backup-now
+ cat
++ echo '01 +1'
++ bc
+ instance=2
+ (( 1 ))
+ (( instance <= backup_instances ))
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ create_zabbix dbbackup4
+ '[' -n dbbackup4 ']'
+ '[' -n '' ']'
+ autoregister=dbbackup4
+ print_debug 'Adding Zabbix Auto Register configuration for '\''dbbackup4'\'''
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/10-db-backup ** [db-backup] Adding Zabbix Auto Register configuration for 'dbbackup4'
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ liftoff
+ output_off
+ '[' true = true ']'
+ set +x
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/99-container ** [container] Skipping '03-monitoring' service for startup routines
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/99-container ** [container] Skipping '04-scheduling' service for startup routines
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/99-container ** [container] Skipping '05-logging' service for startup routines
2023-11-21.11:30:02 [DEBUG] /etc/cont-init.d/99-container ** [container] Skipping '07-firewall' service for startup routines
2023-11-21.11:30:03 [DEBUG] /etc/cont-init.d/99-container ** [container] No Initialization Script Found - Getting ready to start 'dbbackup-01' service
+ PROCESS_NAME=db-backup01-scheduler
+ check_container_initialized
+ print_debug 'Checking to see if container initialization scripts have completed'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:03 [DEBUG] /etc/services.available/dbbackup-01/run ** [db-backup01-scheduler] Checking to see if container initialization scripts have completed
+ output_off
+ '[' true = true ']'
+ set +x
+ check_service_initialized init 10-db-backup
+ print_debug 'Checking to see if service has initialized'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:03 [DEBUG] /etc/services.available/dbbackup-01/run ** [db-backup01-scheduler] Checking to see if service has initialized
+ output_off
+ '[' true = true ']'
+ set +x
+ source /assets/functions/10-db-backup
+ source /assets/defaults/10-db-backup
++ BACKUP_JOB_CONCURRENCY=1
++ DBBACKUP_USER=dbbackup
++ DBBACKUP_GROUP=dbbackup
++ DEFAULT_BACKUP_BEGIN=+0
++ DEFAULT_BACKUP_INTERVAL=1440
++ DEFAULT_BACKUP_INTERVAL=1440
++ DEFAULT_BACKUP_LOCATION=S3
++ DEFAULT_BLOBXFER_REMOTE_PATH=/docker-db-backup
++ DEFAULT_CHECKSUM=MD5
++ DEFAULT_COMPRESSION=GZ
++ DEFAULT_COMPRESSION_LEVEL=3
++ DEFAULT_CREATE_LATEST_SYMLINK=TRUE
++ DEFAULT_ENABLE_PARALLEL_COMPRESSION=TRUE
++ DEFAULT_ENCRYPT=FALSE
++ DEFAULT_FILESYSTEM_PATH=/backup
++ DEFAULT_FILESYSTEM_PATH_PERMISSION=700
++ DEFAULT_FILESYSTEM_PERMISSION=600
++ DEFAULT_FILESYSTEM_ARCHIVE_PATH=/backup/archive/
++ DEFAULT_LOG_LEVEL=DEBUG
++ DEFAULT_MYSQL_ENABLE_TLS=FALSE
++ DEFAULT_MYSQL_EVENTS=TRUE
++ DEFAULT_MYSQL_MAX_ALLOWED_PACKET=512M
++ DEFAULT_MYSQL_SINGLE_TRANSACTION=TRUE
++ DEFAULT_MYSQL_STORED_PROCEDURES=TRUE
++ DEFAULT_MYSQL_TLS_CA_FILE=/etc/ssl/cert.pem
++ DEFAULT_MYSQL_TLS_VERIFY=FALSE
++ DEFAULT_MYSQL_TLS_VERSION=TLSv1.1,TLSv1.2,TLSv1.3
++ DEFAULT_MSSQL_MODE=database
+++ nproc
++ DEFAULT_PARALLEL_COMPRESSION_THREADS=4
++ DEFAULT_RESOURCE_OPTIMIZED=FALSE
++ DEFAULT_S3_CERT_SKIP_VERIFY=TRUE
++ DEFAULT_S3_PROTOCOL=https
++ DEFAULT_SCRIPT_LOCATION_PRE=/assets/scripts/pre/
++ DEFAULT_SCRIPT_LOCATION_POST=/assets/scripts/post/
++ DEFAULT_SIZE_VALUE=bytes
++ DEFAULT_SKIP_AVAILABILITY_CHECK=FALSE
++ DEFAULT_SPLIT_DB=TRUE
++ LOG_PATH=/logs
++ MANUAL_RUN_FOREVER=FALSE
++ MODE=MANUAL
++ TEMP_PATH=/tmp/backups
++ '[' -n '' ']'
+ bootstrap_variables backup_init 01
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ backup_init 01
+ backup_instance_number=01
++ mktemp
+ backup_instance_vars=/tmp/tmp.sUPMMs8ZQg
+ set -o posix
+ set
+ grep -oE '^backup_job_.*='
+ grep -oE '.*='+
sed /--/d
+ read -r backup_instance_var
+ transform_file_var DEFAULT_AUTH DEFAULT_TYPE DEFAULT_HOST DEFAULT_PORT DEFAULT_NAME DEFAULT_NAME_EXCLUDE DEFAULT_USER DEFAULT_PASS DEFAULT_ENCRYPT_PASSPHRASE DEFAULT_ENCRYPT_PUBLIC_KEY DEFAULT_ENCRYPT_PRIVATE_KEY DEFAULT_MONGO_CUSTOM_URI DEFAULT_MYSQL_TLS_CA_FILE DEFAULT_MYSQL_TLS_CERT_FILE DEFAULT_MYSQL_TLS_KEY_FILE DEFAULT_S3_BUCKET DEFAULT_S3_KEY_ID DEFAULT_S3_KEY_SECRET DEFAULT_S3_PATH DEFAULT_S3_REGION DEFAULT_S3_HOST DEFAULT_S3_PROTOCOL DEFAULT_S3_EXTRA_OPTS DEFAULT_S3_CERT_CA_FILE DEFAULT_BLOBXFER_STORAGE_ACCOUNT DEFAULT_BLOBXFER_STORAGE_ACCOUNT_KEY DEFAULT_BLOBXFER_REMOTE_PATH DB01_AUTH DB01_TYPE DB01_HOST DB01_PORT DB01_NAME DB01_NAME_EXCLUDE DB01_USER DB01_PASS DB01_ENCRYPT_PASSPHRASE DB01_ENCRYPT_PUBLIC_KEY DB01_ENCRYPT_PRIVATE_KEY DB01_MONGO_CUSTOM_URI DB01_MYSQL_TLS_CA_FILE DB01_MYSQL_TLS_CERT_FILE DB01_MYSQL_TLS_KEY_FILE DB01_S3_BUCKET DB01_S3_KEY_ID DB01_S3_KEY_SECRET DB01_S3_PATH DB01_S3_REGION DB01_S3_HOST DB01_S3_PROTOCOL DB01_S3_EXTRA_OPTS DB01_S3_CERT_CA_FILE DB01_BLOBXFER_STORAGE_ACCOUNT DB01_BLOBXFER_STORAGE_ACCOUNT_KEY DB01_BLOBXFER_REMOTE_PATH BLOBXFER_STORAGE_ACCOUNT BLOBXFER_STORAGE_KEY DB_HOST DB_NAME DB_PORT DB_USER DB_PASS MONGO_CUSTOM_URI DB_AUTH S3_BUCKET S3_KEY_ID S3_KEY_SECRET S3_PATH S3_REGION S3_HOST S3_PROTOCOL S3_EXTRA_OPTS
+ local variables
++ echo DEFAULT_AUTH DEFAULT_TYPE DEFAULT_HOST DEFAULT_PORT DEFAULT_NAME DEFAULT_NAME_EXCLUDE DEFAULT_USER DEFAULT_PASS DEFAULT_ENCRYPT_PASSPHRASE DEFAULT_ENCRYPT_PUBLIC_KEY DEFAULT_ENCRYPT_PRIVATE_KEY DEFAULT_MONGO_CUSTOM_URI DEFAULT_MYSQL_TLS_CA_FILE DEFAULT_MYSQL_TLS_CERT_FILE DEFAULT_MYSQL_TLS_KEY_FILE DEFAULT_S3_BUCKET DEFAULT_S3_KEY_ID DEFAULT_S3_KEY_SECRET DEFAULT_S3_PATH DEFAULT_S3_REGION DEFAULT_S3_HOST DEFAULT_S3_PROTOCOL DEFAULT_S3_EXTRA_OPTS DEFAULT_S3_CERT_CA_FILE DEFAULT_BLOBXFER_STORAGE_ACCOUNT DEFAULT_BLOBXFER_STORAGE_ACCOUNT_KEY DEFAULT_BLOBXFER_REMOTE_PATH DB01_AUTH DB01_TYPE DB01_HOST DB01_PORT DB01_NAME DB01_NAME_EXCLUDE DB01_USER DB01_PASS DB01_ENCRYPT_PASSPHRASE DB01_ENCRYPT_PUBLIC_KEY DB01_ENCRYPT_PRIVATE_KEY DB01_MONGO_CUSTOM_URI DB01_MYSQL_TLS_CA_FILE DB01_MYSQL_TLS_CERT_FILE DB01_MYSQL_TLS_KEY_FILE DB01_S3_BUCKET DB01_S3_KEY_ID DB01_S3_KEY_SECRET DB01_S3_PATH DB01_S3_REGION DB01_S3_HOST DB01_S3_PROTOCOL DB01_S3_EXTRA_OPTS DB01_S3_CERT_CA_FILE DB01_BLOBXFER_STORAGE_ACCOUNT DB01_BLOBXFER_STORAGE_ACCOUNT_KEY DB01_BLOBXFER_REMOTE_PATH BLOBXFER_STORAGE_ACCOUNT BLOBXFER_STORAGE_KEY DB_HOST DB_NAME DB_PORT DB_USER DB_PASS MONGO_CUSTOM_URI DB_AUTH S3_BUCKET S3_KEY_ID S3_KEY_SECRET S3_PATH S3_REGION S3_HOST S3_PROTOCOL S3_EXTRA_OPTS
++ tr ' ' '\n'
+ variables='DEFAULT_AUTH
DEFAULT_TYPE
DEFAULT_HOST
DEFAULT_PORT
DEFAULT_NAME
DEFAULT_NAME_EXCLUDE
DEFAULT_USER
DEFAULT_PASS
DEFAULT_ENCRYPT_PASSPHRASE
DEFAULT_ENCRYPT_PUBLIC_KEY
DEFAULT_ENCRYPT_PRIVATE_KEY
DEFAULT_MONGO_CUSTOM_URI
DEFAULT_MYSQL_TLS_CA_FILE
DEFAULT_MYSQL_TLS_CERT_FILE
DEFAULT_MYSQL_TLS_KEY_FILE
DEFAULT_S3_BUCKET
DEFAULT_S3_KEY_ID
DEFAULT_S3_KEY_SECRET
DEFAULT_S3_PATH
DEFAULT_S3_REGION
DEFAULT_S3_HOST
DEFAULT_S3_PROTOCOL
DEFAULT_S3_EXTRA_OPTS
DEFAULT_S3_CERT_CA_FILE
DEFAULT_BLOBXFER_STORAGE_ACCOUNT
DEFAULT_BLOBXFER_STORAGE_ACCOUNT_KEY
DEFAULT_BLOBXFER_REMOTE_PATH
DB01_AUTH
DB01_TYPE
DB01_HOST
DB01_PORT
DB01_NAME
DB01_NAME_EXCLUDE
DB01_USER
DB01_PASS
DB01_ENCRYPT_PASSPHRASE
DB01_ENCRYPT_PUBLIC_KEY
DB01_ENCRYPT_PRIVATE_KEY
DB01_MONGO_CUSTOM_URI
DB01_MYSQL_TLS_CA_FILE
DB01_MYSQL_TLS_CERT_FILE
DB01_MYSQL_TLS_KEY_FILE
DB01_S3_BUCKET
DB01_S3_KEY_ID
DB01_S3_KEY_SECRET
DB01_S3_PATH
DB01_S3_REGION
DB01_S3_HOST
DB01_S3_PROTOCOL
DB01_S3_EXTRA_OPTS
DB01_S3_CERT_CA_FILE
DB01_BLOBXFER_STORAGE_ACCOUNT
DB01_BLOBXFER_STORAGE_ACCOUNT_KEY
DB01_BLOBXFER_REMOTE_PATH
BLOBXFER_STORAGE_ACCOUNT
BLOBXFER_STORAGE_KEY
DB_HOST
DB_NAME
DB_PORT
DB_USER
DB_PASS
MONGO_CUSTOM_URI
DB_AUTH
S3_BUCKET
S3_KEY_ID
S3_KEY_SECRET
S3_PATH
S3_REGION
S3_HOST
S3_PROTOCOL
S3_EXTRA_OPTS'
+ for variable in $variables
+ '[' -v DEFAULT_AUTH_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_TYPE_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_HOST_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_PORT_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_NAME_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_NAME_EXCLUDE_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_USER_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_PASS_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_ENCRYPT_PASSPHRASE_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_ENCRYPT_PUBLIC_KEY_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_ENCRYPT_PRIVATE_KEY_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_MONGO_CUSTOM_URI_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_MYSQL_TLS_CA_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_MYSQL_TLS_CERT_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_MYSQL_TLS_KEY_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_BUCKET_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_KEY_ID_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_KEY_SECRET_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_PATH_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_REGION_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_HOST_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_PROTOCOL_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_EXTRA_OPTS_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_S3_CERT_CA_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_BLOBXFER_STORAGE_ACCOUNT_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_BLOBXFER_STORAGE_ACCOUNT_KEY_FILE ']'
+ for variable in $variables
+ '[' -v DEFAULT_BLOBXFER_REMOTE_PATH_FILE ']'
+ for variable in $variables
+ '[' -v DB01_AUTH_FILE ']'
+ for variable in $variables
+ '[' -v DB01_TYPE_FILE ']'
+ for variable in $variables
+ '[' -v DB01_HOST_FILE ']'
+ for variable in $variables
+ '[' -v DB01_PORT_FILE ']'
+ for variable in $variables
+ '[' -v DB01_NAME_FILE ']'
+ for variable in $variables
+ '[' -v DB01_NAME_EXCLUDE_FILE ']'
+ for variable in $variables
+ '[' -v DB01_USER_FILE ']'
+ for variable in $variables
+ '[' -v DB01_PASS_FILE ']'
+ for variable in $variables
+ '[' -v DB01_ENCRYPT_PASSPHRASE_FILE ']'
+ for variable in $variables
+ '[' -v DB01_ENCRYPT_PUBLIC_KEY_FILE ']'
+ for variable in $variables
+ '[' -v DB01_ENCRYPT_PRIVATE_KEY_FILE ']'
+ for variable in $variables
+ '[' -v DB01_MONGO_CUSTOM_URI_FILE ']'
+ for variable in $variables
+ '[' -v DB01_MYSQL_TLS_CA_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DB01_MYSQL_TLS_CERT_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DB01_MYSQL_TLS_KEY_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_BUCKET_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_KEY_ID_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_KEY_SECRET_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_PATH_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_REGION_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_HOST_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_PROTOCOL_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_EXTRA_OPTS_FILE ']'
+ for variable in $variables
+ '[' -v DB01_S3_CERT_CA_FILE_FILE ']'
+ for variable in $variables
+ '[' -v DB01_BLOBXFER_STORAGE_ACCOUNT_FILE ']'
+ for variable in $variables
+ '[' -v DB01_BLOBXFER_STORAGE_ACCOUNT_KEY_FILE ']'
+ for variable in $variables
+ '[' -v DB01_BLOBXFER_REMOTE_PATH_FILE ']'
+ for variable in $variables
+ '[' -v BLOBXFER_STORAGE_ACCOUNT_FILE ']'
+ for variable in $variables
+ '[' -v BLOBXFER_STORAGE_KEY_FILE ']'
+ for variable in $variables
+ '[' -v DB_HOST_FILE ']'
+ for variable in $variables
+ '[' -v DB_NAME_FILE ']'
+ for variable in $variables
+ '[' -v DB_PORT_FILE ']'
+ for variable in $variables
+ '[' -v DB_USER_FILE ']'
+ for variable in $variables
+ '[' -v DB_PASS_FILE ']'
+ for variable in $variables
+ '[' -v MONGO_CUSTOM_URI_FILE ']'
+ for variable in $variables
+ '[' -v DB_AUTH_FILE ']'
+ for variable in $variables
+ '[' -v S3_BUCKET_FILE ']'
+ for variable in $variables
+ '[' -v S3_KEY_ID_FILE ']'
+ for variable in $variables
+ '[' -v S3_KEY_SECRET_FILE ']'
+ for variable in $variables
+ '[' -v S3_PATH_FILE ']'
+ for variable in $variables
+ '[' -v S3_REGION_FILE ']'
+ for variable in $variables
+ '[' -v S3_HOST_FILE ']'
+ for variable in $variables
+ '[' -v S3_PROTOCOL_FILE ']'
+ for variable in $variables
+ '[' -v S3_EXTRA_OPTS_FILE ']'
+ unset file_variable
+ unset variables
+ set -o posix
+ set
+ grep -E '^DB01_|^DEFAULT_|^DB_|^ARCHIVE|^BACKUP_|^BLOBXFER_|^CHECKSUM|^COMPRESSION|^CREATE_|^ENABLE_|^EXTRA_|^GZ_|^INFLUX_|^MYSQL_|^MONGO_|^PARALLEL|^PRE_|^POST_|^S3|^SKIP|^SPLIT'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ '[' -n '' ']'
+ grep -qo '.*_PASS='\''.*'\''' /tmp/tmp.sUPMMs8ZQg
+ grep -qo 'MONGO_CUSTOM_URI='\''.*'\''' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 ARCHIVE_TIME backup_job_archive_time
+ grep -q '^DB01_ARCHIVE_TIME=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_ARCHIVE_TIME=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^ARCHIVE_TIME=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_ARCHIVE_TIME=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 AUTH backup_job_db_auth
+ grep -q '^DB01_AUTH=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DB01_AUTH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' admin '!=' unset ']'
++ grep '^DB01_AUTH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_db_auth=admin
+ backup_job_db_auth=admin
+ transform_backup_instance_variable 01 BACKUP_BEGIN backup_job_backup_begin
+ grep -q '^DB01_BACKUP_BEGIN=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BACKUP_BEGIN=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BACKUP_BEGIN=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BACKUP_BEGIN=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_BACKUP_BEGIN=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' +0 '!=' unset ']'
++ grep '^DEFAULT_BACKUP_BEGIN=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_backup_begin=+0
+ backup_job_backup_begin=+0
+ transform_backup_instance_variable 01 BACKUP_INTERVAL backup_job_backup_interval
+ grep -q '^DB01_BACKUP_INTERVAL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BACKUP_INTERVAL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BACKUP_INTERVAL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BACKUP_INTERVAL=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_BACKUP_INTERVAL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' 1440 '!=' unset ']'
++ grep '^DEFAULT_BACKUP_INTERVAL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_backup_interval=1440
+ backup_job_backup_interval=1440
+ transform_backup_instance_variable 01 BACKUP_GLOBALS backup_job_backup_pgsql_globals
+ grep -q '^DB01_BACKUP_GLOBALS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BACKUP_GLOBALS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BACKUP_GLOBALS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BACKUP_GLOBALS=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 BACKUP_LOCATION backup_job_backup_location
+ grep -q '^DB01_BACKUP_LOCATION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BACKUP_LOCATION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BACKUP_LOCATION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BACKUP_LOCATION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_BACKUP_LOCATION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' S3 '!=' unset ']'
++ grep '^DEFAULT_BACKUP_LOCATION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_backup_location=S3
+ backup_job_backup_location=S3
+ transform_backup_instance_variable 01 BLACKOUT_BEGIN backup_job_snapshot_blackout_start
+ grep -q '^DB01_BLACKOUT_BEGIN=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BLACKOUT_BEGIN=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BLACKOUT_BEGIN=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BLACKOUT_BEGIN=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 BLACKOUT_END backup_job_snapshot_blackout_finish
+ grep -q '^DB01_BLACKOUT_END=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BLACKOUT_END=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BLACKOUT_END=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BLACKOUT_END=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 BLOBXFER_REMOTE_PATH backup_job_blobxfer_remote_path
+ grep -q '^DB01_BLOBXFER_REMOTE_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BLOBXFER_REMOTE_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BLOBXFER_REMOTE_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BLOBXFER_REMOTE_PATH=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_BLOBXFER_REMOTE_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' /docker-db-backup '!=' unset ']'
++ grep '^DEFAULT_BLOBXFER_REMOTE_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_blobxfer_remote_path=/docker-db-backup
+ backup_job_blobxfer_remote_path=/docker-db-backup
+ transform_backup_instance_variable 01 BLOBXFER_STORAGE_ACCOUNT backup_job_blobxfer_storage_account
+ grep -q '^DB01_BLOBXFER_STORAGE_ACCOUNT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BLOBXFER_STORAGE_ACCOUNT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BLOBXFER_STORAGE_ACCOUNT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BLOBXFER_STORAGE_ACCOUNT=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 BLOBXFER_STORAGE_KEY backup_job_blobxfer_storage_key
+ grep -q '^DB01_BLOBXFER_STORAGE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_BLOBXFER_STORAGE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^BLOBXFER_STORAGE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_BLOBXFER_STORAGE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 CHECKSUM backup_job_checksum
+ grep -q '^DB01_CHECKSUM=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_CHECKSUM=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^CHECKSUM=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_CHECKSUM=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_CHECKSUM=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' MD5 '!=' unset ']'
++ grep '^DEFAULT_CHECKSUM=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_checksum=MD5
+ backup_job_checksum=MD5
+ transform_backup_instance_variable 01 CLEANUP_TIME backup_job_cleanup_time
+ grep -q '^DB01_CLEANUP_TIME=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_CLEANUP_TIME=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^CLEANUP_TIME=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_CLEANUP_TIME=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 COMPRESSION backup_job_compression
+ grep -q '^DB01_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' GZ '!=' unset ']'
++ grep '^DEFAULT_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_compression=GZ
+ backup_job_compression=GZ
+ transform_backup_instance_variable 01 COMPRESSION_LEVEL backup_job_compression_level
+ grep -q '^DB01_COMPRESSION_LEVEL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_COMPRESSION_LEVEL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^COMPRESSION_LEVEL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_COMPRESSION_LEVEL=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_COMPRESSION_LEVEL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' 3 '!=' unset ']'
++ grep '^DEFAULT_COMPRESSION_LEVEL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_compression_level=3
+ backup_job_compression_level=3
+ transform_backup_instance_variable 01 CREATE_LATEST_SYMLINK backup_job_create_latest_symlink
+ grep -q '^DB01_CREATE_LATEST_SYMLINK=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_CREATE_LATEST_SYMLINK=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^CREATE_LATEST_SYMLINK=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_CREATE_LATEST_SYMLINK=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_CREATE_LATEST_SYMLINK=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TRUE '!=' unset ']'
++ grep '^DEFAULT_CREATE_LATEST_SYMLINK=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_create_latest_symlink=TRUE
+ backup_job_create_latest_symlink=TRUE
+ transform_backup_instance_variable 01 ENABLE_PARALLEL_COMPRESSION backup_job_parallel_compression
+ grep -q '^DB01_ENABLE_PARALLEL_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_ENABLE_PARALLEL_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^ENABLE_PARALLEL_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_ENABLE_PARALLEL_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_ENABLE_PARALLEL_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TRUE '!=' unset ']'
++ grep '^DEFAULT_ENABLE_PARALLEL_COMPRESSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_parallel_compression=TRUE
+ backup_job_parallel_compression=TRUE
+ transform_backup_instance_variable 01 ENCRYPT backup_job_encrypt
+ grep -q '^DB01_ENCRYPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_ENCRYPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^ENCRYPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_ENCRYPT=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_ENCRYPT=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' FALSE '!=' unset ']'
++ grep '^DEFAULT_ENCRYPT=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_encrypt=FALSE
+ backup_job_encrypt=FALSE
+ transform_backup_instance_variable 01 ENCRYPT_PASSPHRASE backup_job_encrypt_passphrase
+ grep -q '^DB01_ENCRYPT_PASSPHRASE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_ENCRYPT_PASSPHRASE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^ENCRYPT_PASSPHRASE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_ENCRYPT_PASSPHRASE=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 ENCRYPT_PRIVATE_KEY backup_job_encrypt_private_key
+ grep -q '^DB01_ENCRYPT_PRIVATE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_ENCRYPT_PRIVATE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^ENCRYPT_PRIVATE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_ENCRYPT_PRIVATE_KEY=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 ENCRYPT_PUBLIC_KEY backup_job_encrypt_public_key
+ grep -q '^DB01_ENCRYPT_PUBLIC_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_ENCRYPT_PUBLIC_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^ENCRYPT_PUBLIC_KEY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_ENCRYPT_PUBLIC_KEY=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 EXTRA_DUMP_OPTS backup_job_extra_dump_opts
+ grep -q '^DB01_EXTRA_DUMP_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_EXTRA_DUMP_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^EXTRA_DUMP_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_EXTRA_DUMP_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 EXTRA_ENUMERATION_OPTS backup_job_extra_enumeration_opts
+ grep -q '^DB01_EXTRA_ENUMERATION_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_EXTRA_ENUMERATION_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^EXTRA_ENUMERATION_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_EXTRA_ENUMERATION_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 EXTRA_OPTS backup_job_extra_opts
+ grep -q '^DB01_EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 FILESYSTEM_ARCHIVE_PATH backup_job_filesystem_archive_path
+ grep -q '^DB01_FILESYSTEM_ARCHIVE_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_FILESYSTEM_ARCHIVE_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^FILESYSTEM_ARCHIVE_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_FILESYSTEM_ARCHIVE_PATH=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_FILESYSTEM_ARCHIVE_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' /backup/archive/ '!=' unset ']'
++ grep '^DEFAULT_FILESYSTEM_ARCHIVE_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_filesystem_archive_path=/backup/archive/
+ backup_job_filesystem_archive_path=/backup/archive/
+ transform_backup_instance_variable 01 FILESYSTEM_PATH backup_job_filesystem_path
+ grep -q '^DB01_FILESYSTEM_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_FILESYSTEM_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^FILESYSTEM_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_FILESYSTEM_PATH=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_FILESYSTEM_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' /backup '!=' unset ']'
++ grep '^DEFAULT_FILESYSTEM_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_filesystem_path=/backup
+ backup_job_filesystem_path=/backup
+ transform_backup_instance_variable 01 FILESYSTEM_PATH_PERMISSION backup_job_filesystem_path_permission
+ grep -q '^DB01_FILESYSTEM_PATH_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_FILESYSTEM_PATH_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^FILESYSTEM_PATH_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_FILESYSTEM_PATH_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_FILESYSTEM_PATH_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' 700 '!=' unset ']'
++ grep '^DEFAULT_FILESYSTEM_PATH_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_filesystem_path_permission=700
+ backup_job_filesystem_path_permission=700
+ transform_backup_instance_variable 01 FILESYSTEM_PERMISSION backup_job_filesystem_permission
+ grep -q '^DB01_FILESYSTEM_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_FILESYSTEM_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^FILESYSTEM_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_FILESYSTEM_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_FILESYSTEM_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' 600 '!=' unset ']'
++ grep '^DEFAULT_FILESYSTEM_PERMISSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_filesystem_permission=600
+ backup_job_filesystem_permission=600
+ transform_backup_instance_variable 01 GZ_RSYNCABLE backup_job_gz_rsyncable
+ grep -q '^DB01_GZ_RSYNCABLE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_GZ_RSYNCABLE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^GZ_RSYNCABLE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_GZ_RSYNCABLE=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 HOST backup_job_db_host
+ grep -q '^DB01_HOST=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DB01_HOST=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' mongodb-headless '!=' unset ']'
++ grep '^DB01_HOST=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_db_host=mongodb-headless
+ backup_job_db_host=mongodb-headless
+ transform_backup_instance_variable 01 INFLUX_VERSION backup_job_influx_version
+ grep -q '^DB01_INFLUX_VERSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_INFLUX_VERSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^INFLUX_VERSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_INFLUX_VERSION=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 LOG_LEVEL backup_job_log_level
+ grep -q '^DB01_LOG_LEVEL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_LOG_LEVEL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^LOG_LEVEL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_LOG_LEVEL=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_LOG_LEVEL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' DEBUG '!=' unset ']'
++ grep '^DEFAULT_LOG_LEVEL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_log_level=DEBUG
+ backup_job_log_level=DEBUG
+ transform_backup_instance_variable 01 MONGO_CUSTOM_URI backup_job_mongo_custom_uri
+ grep -q '^DB01_MONGO_CUSTOM_URI=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MONGO_CUSTOM_URI=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MONGO_CUSTOM_URI=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MONGO_CUSTOM_URI=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 MYSQL_ENABLE_TLS backup_job_mysql_enable_tls
+ grep -q '^DB01_MYSQL_ENABLE_TLS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_ENABLE_TLS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_ENABLE_TLS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_ENABLE_TLS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
++ grep '^DEFAULT_MYSQL_ENABLE_TLS=' /tmp/tmp.sUPMMs8ZQg
+ '[' FALSE '!=' unset ']'
++ grep '^DEFAULT_MYSQL_ENABLE_TLS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_mysql_enable_tls=FALSE
+ backup_job_mysql_enable_tls=FALSE
+ transform_backup_instance_variable 01 MYSQL_EVENTS backup_job_mysql_events
+ grep -q '^DB01_MYSQL_EVENTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_EVENTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_EVENTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_EVENTS=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MYSQL_EVENTS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TRUE '!=' unset ']'
++ grep '^DEFAULT_MYSQL_EVENTS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_mysql_events=TRUE
+ backup_job_mysql_events=TRUE
+ transform_backup_instance_variable 01 MYSQL_MAX_ALLOWED_PACKET backup_job_mysql_max_allowed_packet
+ grep -q '^DB01_MYSQL_MAX_ALLOWED_PACKET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_MAX_ALLOWED_PACKET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_MAX_ALLOWED_PACKET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_MAX_ALLOWED_PACKET=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MYSQL_MAX_ALLOWED_PACKET=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' 512M '!=' unset ']'
++ grep '^DEFAULT_MYSQL_MAX_ALLOWED_PACKET=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_mysql_max_allowed_packet=512M
+ backup_job_mysql_max_allowed_packet=512M
+ transform_backup_instance_variable 01 MYSQL_SINGLE_TRANSACTION backup_job_mysql_single_transaction
+ grep -q '^DB01_MYSQL_SINGLE_TRANSACTION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_SINGLE_TRANSACTION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_SINGLE_TRANSACTION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_SINGLE_TRANSACTION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MYSQL_SINGLE_TRANSACTION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TRUE '!=' unset ']'
++ grep '^DEFAULT_MYSQL_SINGLE_TRANSACTION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_mysql_single_transaction=TRUE
+ backup_job_mysql_single_transaction=TRUE
+ transform_backup_instance_variable 01 MYSQL_STORED_PROCEDURES backup_job_mysql_stored_procedures
+ grep -q '^DB01_MYSQL_STORED_PROCEDURES=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_STORED_PROCEDURES=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_STORED_PROCEDURES=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_STORED_PROCEDURES=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MYSQL_STORED_PROCEDURES=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TRUE '!=' unset ']'
++ cut -d = -f2-
++ grep '^DEFAULT_MYSQL_STORED_PROCEDURES=' /tmp/tmp.sUPMMs8ZQg
+ export backup_job_mysql_stored_procedures=TRUE
+ backup_job_mysql_stored_procedures=TRUE
+ transform_backup_instance_variable 01 MYSQL_TLS_CA_FILE backup_job_mysql_tls_ca_file
+ grep -q '^DB01_MYSQL_TLS_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_TLS_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_TLS_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_TLS_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MYSQL_TLS_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' /etc/ssl/cert.pem '!=' unset ']'
++ cut -d = -f2-
++ grep '^DEFAULT_MYSQL_TLS_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ export backup_job_mysql_tls_ca_file=/etc/ssl/cert.pem
+ backup_job_mysql_tls_ca_file=/etc/ssl/cert.pem
+ transform_backup_instance_variable 01 MYSQL_TLS_CERT_FILE backup_job_mysql_tls_cert_file
+ grep -q '^DB01_MYSQL_TLS_CERT_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_TLS_CERT_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_TLS_CERT_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_TLS_CERT_FILE=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 MYSQL_TLS_KEY_FILE backup_job_mysql_tls_key_file
+ grep -q '^DB01_MYSQL_TLS_KEY_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_TLS_KEY_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_TLS_KEY_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_TLS_KEY_FILE=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 MYSQL_TLS_VERIFY backup_job_mysql_tls_verify
+ grep -q '^DB01_MYSQL_TLS_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_TLS_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_TLS_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_TLS_VERIFY=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MYSQL_TLS_VERIFY=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' FALSE '!=' unset ']'
++ grep '^DEFAULT_MYSQL_TLS_VERIFY=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_mysql_tls_verify=FALSE
+ backup_job_mysql_tls_verify=FALSE
+ transform_backup_instance_variable 01 MYSQL_TLS_VERSION backup_job_mysql_tls_version
+ grep -q '^DB01_MYSQL_TLS_VERSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MYSQL_TLS_VERSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MYSQL_TLS_VERSION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MYSQL_TLS_VERSION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MYSQL_TLS_VERSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TLSv1.1,TLSv1.2,TLSv1.3 '!=' unset ']'
++ grep '^DEFAULT_MYSQL_TLS_VERSION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_mysql_tls_version=TLSv1.1,TLSv1.2,TLSv1.3
+ backup_job_mysql_tls_version=TLSv1.1,TLSv1.2,TLSv1.3
+ transform_backup_instance_variable 01 MSSQL_MODE backup_job_mssql_mode
+ grep -q '^DB01_MSSQL_MODE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_MSSQL_MODE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^MSSQL_MODE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_MSSQL_MODE=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_MSSQL_MODE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' database '!=' unset ']'
++ grep '^DEFAULT_MSSQL_MODE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_mssql_mode=database
+ backup_job_mssql_mode=database
+ transform_backup_instance_variable 01 NAME backup_job_db_name
+ grep -q '^DB01_NAME=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DB01_NAME=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' base_monitor '!=' unset ']'
++ grep '^DB01_NAME=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_db_name=base_monitor
+ backup_job_db_name=base_monitor
+ transform_backup_instance_variable 01 NAME_EXCLUDE backup_job_db_name_exclude
+ grep -q '^DB01_NAME_EXCLUDE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_NAME_EXCLUDE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^NAME_EXCLUDE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_NAME_EXCLUDE=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 PARALLEL_COMPRESSION_THREADS backup_job_parallel_compression_threads
+ grep -q '^DB01_PARALLEL_COMPRESSION_THREADS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_PARALLEL_COMPRESSION_THREADS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^PARALLEL_COMPRESSION_THREADS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_PARALLEL_COMPRESSION_THREADS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
++ grep '^DEFAULT_PARALLEL_COMPRESSION_THREADS=' /tmp/tmp.sUPMMs8ZQg
+ '[' 4 '!=' unset ']'
++ grep '^DEFAULT_PARALLEL_COMPRESSION_THREADS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_parallel_compression_threads=4
+ backup_job_parallel_compression_threads=4
+ transform_backup_instance_variable 01 PASS backup_job_db_pass
+ grep -q '^DB01_PASS=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DB01_PASS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' dummypass '!=' unset ']'
++ grep '^DB01_PASS=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_db_pass=dummypass
+ backup_job_db_pass=dummypass
+ transform_backup_instance_variable 01 PORT backup_job_db_port
+ grep -q '^DB01_PORT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_PORT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^PORT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_PORT=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 POST_SCRIPT_X_VERIFY backup_job_post_script_x_verify
+ grep -q '^DB01_POST_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_POST_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^POST_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_POST_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 POST_SCRIPT backup_job_post_script
+ grep -q '^DB01_POST_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_POST_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^POST_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_POST_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 PRE_SCRIPT backup_job_pre_script
+ grep -q '^DB01_PRE_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_PRE_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^PRE_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_PRE_SCRIPT=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 PRE_SCRIPT_X_VERIFY backup_job_pre_script_x_verify
+ grep -q '^DB01_PRE_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_PRE_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^PRE_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_PRE_SCRIPT_X_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 RESOURCE_OPTIMIZED backup_job_resource_optimized
+ grep -q '^DB01_RESOURCE_OPTIMIZED=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_RESOURCE_OPTIMIZED=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^RESOURCE_OPTIMIZED=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_RESOURCE_OPTIMIZED=' /tmp/tmp.sUPMMs8ZQg
++ grep ++ '^DEFAULT_RESOURCE_OPTIMIZED=' cut -d = -f2
/tmp/tmp.sUPMMs8ZQg
+ '[' FALSE '!=' unset ']'
++ grep ++ '^DEFAULT_RESOURCE_OPTIMIZED=' cut -d = -f2-
/tmp/tmp.sUPMMs8ZQg
+ export backup_job_resource_optimized=FALSE
+ backup_job_resource_optimized=FALSE
+ transform_backup_instance_variable 01 S3_BUCKET backup_job_s3_bucket
+ grep -q '^DB01_S3_BUCKET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_BUCKET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_BUCKET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_BUCKET=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_BUCKET=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' fabric-dbbackups-prod '!=' unset ']'
++ cut -d = -f2-
++ grep '^DEFAULT_S3_BUCKET=' /tmp/tmp.sUPMMs8ZQg
+ export backup_job_s3_bucket=fabric-dbbackups-prod
+ backup_job_s3_bucket=fabric-dbbackups-prod
+ transform_backup_instance_variable 01 S3_CERT_CA_FILE backup_job_s3_cert_ca_file
+ grep -q '^DB01_S3_CERT_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_CERT_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_CERT_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_CERT_CA_FILE=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 S3_CERT_SKIP_VERIFY backup_job_s3_cert_skip_verify
+ grep -q '^DB01_S3_CERT_SKIP_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_CERT_SKIP_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_CERT_SKIP_VERIFY=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_CERT_SKIP_VERIFY=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_CERT_SKIP_VERIFY=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TRUE '!=' unset ']'
++ grep '^DEFAULT_S3_CERT_SKIP_VERIFY=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_s3_cert_skip_verify=TRUE
+ backup_job_s3_cert_skip_verify=TRUE
+ transform_backup_instance_variable 01 S3_EXTRA_OPTS backup_job_s3_extra_opts
+ grep -q '^DB01_S3_EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_EXTRA_OPTS=' /tmp/tmp.sUPMMs8ZQg
+ transform_backup_instance_variable 01 S3_HOST backup_job_s3_host
+ grep -q '^DB01_S3_HOST=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_HOST=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_HOST=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_HOST=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_HOST=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' storage.googleapis.com '!=' unset ']'
++ cut -d = -f2-
++ grep '^DEFAULT_S3_HOST=' /tmp/tmp.sUPMMs8ZQg
+ export backup_job_s3_host=storage.googleapis.com
+ backup_job_s3_host=storage.googleapis.com
+ transform_backup_instance_variable 01 S3_KEY_ID backup_job_s3_key_id
+ grep -q '^DB01_S3_KEY_ID=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_KEY_ID=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_KEY_ID=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_KEY_ID=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_KEY_ID=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' KEY_ID '!=' unset ']'
++ grep '^DEFAULT_S3_KEY_ID=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_s3_key_id=KEY_ID
+ backup_job_s3_key_id=KEY_ID
+ transform_backup_instance_variable 01 S3_KEY_SECRET backup_job_s3_key_secret
+ grep -q '^DB01_S3_KEY_SECRET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_KEY_SECRET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_KEY_SECRET=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_KEY_SECRET=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_KEY_SECRET=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' KEY_SECRET '!=' unset ']'
++ grep '^DEFAULT_S3_KEY_SECRET=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_s3_key_secret=KEY_SECRET
+ backup_job_s3_key_secret=KEY_SECRET
+ transform_backup_instance_variable 01 S3_PATH backup_job_s3_path
+ grep -q '^DB01_S3_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_PATH=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_PATH=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' GMFC-RKD/2023/11/21 '!=' unset ']'
++ grep '^DEFAULT_S3_PATH=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_s3_path=GMFC-RKD/2023/11/21
+ backup_job_s3_path=GMFC-RKD/2023/11/21
+ transform_backup_instance_variable 01 S3_PROTOCOL backup_job_s3_protocol
+ grep -q '^DB01_S3_PROTOCOL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_PROTOCOL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_PROTOCOL=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_PROTOCOL=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_PROTOCOL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' https '!=' unset ']'
++ grep '^DEFAULT_S3_PROTOCOL=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_s3_protocol=https
+ backup_job_s3_protocol=https
+ transform_backup_instance_variable 01 S3_REGION backup_job_s3_region
+ grep -q '^DB01_S3_REGION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_S3_REGION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^S3_REGION=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_S3_REGION=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_S3_REGION=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' us '!=' unset ']'
++ grep '^DEFAULT_S3_REGION=' /tmp/tmp.sUPMMs8ZQg++ cut -d = -f2-

+ export backup_job_s3_region=us
+ backup_job_s3_region=us
+ transform_backup_instance_variable 01 SCRIPT_LOCATION_POST backup_job_script_location_post
+ grep -q '^DB01_SCRIPT_LOCATION_POST=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_SCRIPT_LOCATION_POST=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^SCRIPT_LOCATION_POST=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_SCRIPT_LOCATION_POST=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_SCRIPT_LOCATION_POST=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' /assets/scripts/post/ '!=' unset ']'
++ grep '^DEFAULT_SCRIPT_LOCATION_POST=' ++ cut -d = -f2-
/tmp/tmp.sUPMMs8ZQg
+ export backup_job_script_location_post=/assets/scripts/post/
+ backup_job_script_location_post=/assets/scripts/post/
+ transform_backup_instance_variable 01 SCRIPT_LOCATION_PRE backup_job_script_location_pre
+ grep -q '^DB01_SCRIPT_LOCATION_PRE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_SCRIPT_LOCATION_PRE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^SCRIPT_LOCATION_PRE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_SCRIPT_LOCATION_PRE=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_SCRIPT_LOCATION_PRE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' /assets/scripts/pre/ '!=' unset ']'
++ grep '^DEFAULT_SCRIPT_LOCATION_PRE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_script_location_pre=/assets/scripts/pre/
+ backup_job_script_location_pre=/assets/scripts/pre/
+ transform_backup_instance_variable 01 SIZE_VALUE backup_job_size_value
+ grep -q '^DB01_SIZE_VALUE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_SIZE_VALUE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^SIZE_VALUE=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_SIZE_VALUE=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_SIZE_VALUE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' bytes '!=' unset ']'
++ grep '^DEFAULT_SIZE_VALUE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_size_value=bytes
+ backup_job_size_value=bytes
+ transform_backup_instance_variable 01 SKIP_AVAILABILITY_CHECK backup_job_skip_availability_check
+ grep -q '^DB01_SKIP_AVAILABILITY_CHECK=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_SKIP_AVAILABILITY_CHECK=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^SKIP_AVAILABILITY_CHECK=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_SKIP_AVAILABILITY_CHECK=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_SKIP_AVAILABILITY_CHECK=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' FALSE '!=' unset ']'
++ grep '^DEFAULT_SKIP_AVAILABILITY_CHECK=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_skip_availability_check=FALSE
+ backup_job_skip_availability_check=FALSE
+ transform_backup_instance_variable 01 SPLIT_DB backup_job_split_db
+ grep -q '^DB01_SPLIT_DB=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DB_SPLIT_DB=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^SPLIT_DB=' /tmp/tmp.sUPMMs8ZQg
+ grep -q '^DEFAULT_SPLIT_DB=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DEFAULT_SPLIT_DB=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' TRUE '!=' unset ']'
++ grep '^DEFAULT_SPLIT_DB=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_split_db=TRUE
+ backup_job_split_db=TRUE
+ transform_backup_instance_variable 01 TYPE backup_job_db_type
+ grep -q '^DB01_TYPE=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DB01_TYPE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' mongo '!=' unset ']'
++ grep '^DB01_TYPE=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_db_type=mongo
+ backup_job_db_type=mongo
+ transform_backup_instance_variable 01 USER backup_job_db_user
+ grep -q '^DB01_USER=' /tmp/tmp.sUPMMs8ZQg
++ grep '^DB01_USER=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2
+ '[' csr '!=' unset ']'
++ grep '^DB01_USER=' /tmp/tmp.sUPMMs8ZQg
++ cut -d = -f2-
+ export backup_job_db_user=csr
+ backup_job_db_user=csr
++ echo +0
++ sed -e 's|'\''||g' -e 's|"||g'
+ backup_job_backup_begin=+0
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ rm -rf /tmp/tmp.sUPMMs8ZQg
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ bootstrap_variables parse_variables 01
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ parse_variables 01
+ local v_instance=01
+ check_var backup_job_db_type DB01_TYPE 'appropriate database type'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:03 [DEBUG] /etc/services.available/dbbackup-01/run ** [db-backup01-scheduler] [parse_variables] Looking for existence of DB01_TYPE environment variable
+ '[' '!' -v backup_job_db_type ']'
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ case "${backup_job_db_type,,}" in
+ dbtype=mongo
+ '[' -n '' ']'
+ backup_job_db_port=27017
+ [[ -n csr ]]
+ MONGO_USER_STR=' --username csr'
+ [[ -n dummypass ]]
+ MONGO_PASS_STR=' --password dummypass'
+ [[ -n base_monitor ]]
+ MONGO_DB_STR=' --db base_monitor'
+ [[ -n admin ]]
+ MONGO_AUTH_STR=' --authenticationDatabase admin'
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' -z base_monitor ']'
+ PROCESS_NAME=01-mongodb-headless__base_monitor
+ trap ctrl_c INT
+ [[ manual =~ standalone ]]
+ '[' dbbackup-01 = manual ']'
+ '[' dbbackup-01 = now ']'
+ silent sleep 01
+ '[' TRUE = true ']'
+ '[' '' = true ']'
+ '[' debug = debug ']'
+ sleep 01
+ time_last_run=0
++ date +%s
+ time_current=1700566204
+ [[ +0 =~ ^\+(.*)$ ]]
+ print_debug 'BACKUP_BEGIN is a jump of minute starting with +'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:04 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] BACKUP_BEGIN is a jump of minute starting with +
+ timer plusvalue
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ time_wait=0
+ time_future=1700566204
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ print_debug 'Wait Time: 0 Future execution time: 1700566204 Current Time: 1700566204'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:04 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Wait Time: 0 Future execution time: 1700566204 Current Time: 1700566204
++ date -d @1700566204 '+%Y-%m-%d %T %Z'
+ print_info 'Next Backup at 2023-11-21 11:30:04 UTC'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:04 [INFO] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Next Backup at 2023-11-21 11:30:04 UTC
+ silent sleep 0
+ '[' TRUE = true ']'
+ '[' '' = true ']'
+ '[' debug = debug ']'
+ sleep 0
+ true
+ '[' -n '' ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ timer job start
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ case "${2}" in
++ date +%s
+ backup_job_start_time=1700566205
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ process_limiter
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ true
+ counter=0
++ awk '{print $1}'
++ wc -l /tmp/.container/db-backup-backups
+ process_amount=0
+ '[' 0 -ge 1 ']'
+ break
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ echo 01
++ date '+%Y-%m-%d %T %Z'
+ print_debug 'Backup 01 routines started time: 2023-11-21 11:30:05 UTC'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Backup 01 routines started time: 2023-11-21 11:30:05 UTC
+ bootstrap_filesystem
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' '!' -d /backup ']'
+ mkdir -p /backup
++ stat -c %U /backup
+ '[' root '!=' dbbackup ']'
+ chown -R dbbackup:dbbackup /backup
++ stat -c %a /backup
+ '[' 755 '!=' 700 ']'
+ chmod 700 /backup
+ '[' -d /backup/archive/ ']'
+ '[' '!' -d /logs ']'
+ mkdir -p /logs
++ stat -c %U /logs
+ '[' root '!=' dbbackup ']'
+ chown -R dbbackup:dbbackup /logs
++ date +%Y%m%d
+ '[' '!' -d /logs/20231121 ']'
++ date +%Y%m%d
+ run_as_user mkdir -p /logs/20231121
+ sudo -Eu dbbackup mkdir -p /logs/20231121
++ stat -c %a /logs
+ '[' 755 '!=' 755 ']'
+ '[' '!' -d /tmp/backups ']'
+ mkdir -p /tmp/backups
++ stat -c %U /tmp/backups
+ '[' root '!=' dbbackup ']'
+ chown -R dbbackup:dbbackup /tmp/backups
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ check_availability
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_false FALSE
+ '[' false = false ']'
+ case "${dbtype}" in
+ '[' -n '' ']'
+ counter=0
+ run_as_user nc -z mongodb-headless 27017
+ sudo -Eu dbbackup nc -z mongodb-headless 27017
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ backup_mongo
+ prepare_dbbackup
+ timer backup start
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ case "${2}" in
++ run_as_user date +%s
++ sudo -Eu dbbackup date +%s
+ dbbackup_start_time=1700566205
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
++ run_as_user date +%Y%m%d-%H%M%S
++ sudo -Eu dbbackup date +%Y%m%d-%H%M%S
+ now=20231121-113005
+ backup_job_filename_base=mongo_base_monitor_mongodb-headless
+ backup_job_filename=mongo_base_monitor_mongodb-headless_20231121-113005.sql
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' gz = none ']'
+ backup_job_filename=mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ backup_job_filename_base=mongo_base_monitor_mongodb-headless
+ mongo_compression=--gzip
+ compression_string='and compressing with gzip'
+ '[' -n '' ']'
+ mongo_backup_parameter='--host mongodb-headless --port 27017  --username csr --password dummypass --authenticationDatabase admin --db base_monitor  '
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ pre_dbbackup base_monitor
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' -n '' ']'
+ '[' -d /assets/custom-scripts/pre ']'
+ '[' -d /assets/scripts/pre/ ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ write_log notice 'Dumping MongoDB database: '\'''\'' and compressing with gzip'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Dumping MongoDB database: '' and compressing with gzip
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ silent run_as_user mongodump --archive=/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz --gzip --host mongodb-headless --port 27017 --username csr --password dummypass --authenticationDatabase admin --db base_monitor
+ '[' TRUE = true ']'
+ '[' '' = true ']'
+ '[' debug = debug ']'
+ run_as_user mongodump --archive=/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz --gzip --host mongodb-headless --port 27017 --username csr --password dummypass --authenticationDatabase admin --db base_monitor
+ sudo -Eu dbbackup mongodump --archive=/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz --gzip --host mongodb-headless --port 27017 --username csr --password dummypass --authenticationDatabase admin --db base_monitor
2023-11-21T11:30:05.319+0000    writing base_monitor.sticker_densities to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.321+0000    writing base_monitor.stickers_occupancies_updates to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.324+0000    writing base_monitor.sticker_updates to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.333+0000    done dumping base_monitor.sticker_densities (0 documents)
2023-11-21T11:30:05.334+0000    writing base_monitor.general_status to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.334+0000    done dumping base_monitor.stickers_occupancies_updates (0 documents)
2023-11-21T11:30:05.335+0000    done dumping base_monitor.sticker_updates (0 documents)
2023-11-21T11:30:05.335+0000    writing base_monitor.robots_last_update to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.335+0000    writing base_monitor.lifts_slots_updates to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.337+0000    writing base_monitor.investigation_snapshots to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.346+0000    done dumping base_monitor.robots_last_update (29 documents)
2023-11-21T11:30:05.349+0000    writing base_monitor.cell_updates to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.371+0000    done dumping base_monitor.cell_updates (0 documents)
2023-11-21T11:30:05.373+0000    writing base_monitor.stations to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.379+0000    done dumping base_monitor.investigation_snapshots (0 documents)
2023-11-21T11:30:05.379+0000    writing base_monitor.tray_stats to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.385+0000    done dumping base_monitor.lifts_slots_updates (0 documents)
2023-11-21T11:30:05.386+0000    writing base_monitor.robots to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.388+0000    done dumping base_monitor.general_status (0 documents)
2023-11-21T11:30:05.388+0000    writing base_monitor.tile_guards_updates to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.389+0000    done dumping base_monitor.robots (0 documents)
2023-11-21T11:30:05.390+0000    writing base_monitor.cell_densities to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.394+0000    done dumping base_monitor.tile_guards_updates (0 documents)
2023-11-21T11:30:05.394+0000    done dumping base_monitor.cell_densities (0 documents)
2023-11-21T11:30:05.395+0000    writing base_monitor.aborted_missions to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.395+0000    writing base_monitor.aisles_update to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.404+0000    done dumping base_monitor.aisles_update (0 documents)
2023-11-21T11:30:05.405+0000    done dumping base_monitor.stations (0 documents)
2023-11-21T11:30:05.405+0000    writing base_monitor.lifts_responsibility_areas to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.407+0000    writing base_monitor.pending_discrepancy_fix to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.414+0000    done dumping base_monitor.tray_stats (0 documents)
2023-11-21T11:30:05.415+0000    writing base_monitor.plan_status to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.417+0000    done dumping base_monitor.lifts_responsibility_areas (0 documents)
2023-11-21T11:30:05.419+0000    done dumping base_monitor.pending_discrepancy_fix (0 documents)
2023-11-21T11:30:05.422+0000    writing base_monitor.plan_debug to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.425+0000    writing base_monitor.sticky_notes_history to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.425+0000    done dumping base_monitor.aborted_missions (0 documents)
2023-11-21T11:30:05.427+0000    writing base_monitor.sticker_guards_updates to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.445+0000    done dumping base_monitor.plan_status (0 documents)
2023-11-21T11:30:05.445+0000    writing base_monitor.disabled_meetups to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.452+0000    done dumping base_monitor.plan_debug (0 documents)
2023-11-21T11:30:05.455+0000    writing base_monitor.aisles_updates_history to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.458+0000    done dumping base_monitor.disabled_meetups (0 documents)
2023-11-21T11:30:05.459+0000    writing base_monitor.robot_sticker_occupancies_updates to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.461+0000    done dumping base_monitor.sticky_notes_history (0 documents)
2023-11-21T11:30:05.469+0000    writing base_monitor.stations_and_robots_discovery to archive '/tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
2023-11-21T11:30:05.472+0000    done dumping base_monitor.sticker_guards_updates (0 documents)
2023-11-21T11:30:05.472+0000    done dumping base_monitor.aisles_updates_history (0 documents)
2023-11-21T11:30:05.476+0000    done dumping base_monitor.robot_sticker_occupancies_updates (0 documents)
2023-11-21T11:30:05.480+0000    done dumping base_monitor.stations_and_robots_discovery (0 documents)
+ exit_code=0
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ check_exit_code backup mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ write_log debug 'DB Backup exit Code is 0'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] DB Backup exit Code is 0
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${exit_code}" in
+ write_log info 'DB Backup of '\''mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'\'' completed successfully'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [INFO] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] DB Backup of 'mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz' completed successfully
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ timer backup finish
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ case "${2}" in
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ file_encryption
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ generate_checksum
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' 0 = 0 ']'
+ case "${backup_job_checksum,,}" in
+ checksum_command=' md5sum'
+ checksum_extension=md5
++ date +%s
+ checksum_routines_start_time=1700566205
+ write_log notice 'Generating MD5 sum for '\''mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'\'''
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Generating MD5 sum for 'mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ cd /tmp/backups
+ run_as_user md5sum mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ run_as_user tee mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
+ sudo -Eu dbbackup md5sum mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ sudo -Eu dbbackup tee mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
+ chmod 600 mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
++ run_as_user cat mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
++ ++ sudo -Eu dbbackup cat mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
awk '{print $1}'
+ checksum_value=e2948875adac263f18f8df7f693a27a6
++ date +%s
+ checksum_routines_finish_time=1700566205
++ echo 0
+ checksum_routines_total_time=0
++ cat
+ zabbix_checksum_time='- dbbackup.backup.checksum.duration.[mongodb-headless.base_monitor] 0
- dbbackup.backup.checksum.hash.[mongodb-headless.base_monitor] e2948875adac263f18f8df7f693a27a6'
+ write_log debug 'MD5: e2948875adac263f18f8df7f693a27a6 - mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] MD5: e2948875adac263f18f8df7f693a27a6 - mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
++ echo 0
++ awk '{printf "Hours: %d Minutes: %02d Seconds: %02d", $1/3600, ($1/60)%60, $1%60}'
+ write_log debug 'Checksum routines time taken: Hours: 0 Minutes: 00 Seconds: 00'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Checksum routines time taken: Hours: 0 Minutes: 00 Seconds: 00
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ move_dbbackup
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' 0 = 0 ']'
++ run_as_user stat -c%s /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
++ sudo -Eu dbbackup stat -c%s /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ dbbackup_size=7213
++ run_as_user date -r /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz +%s
++ sudo -Eu dbbackup date -r /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz +%s
+ dbbackup_date=1700566205
+ case "${backup_job_size_value,,}" in
+ backup_job_size_value=1
+ '[' 1 = 1 ']'
++ run_as_user stat -c%s /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
++ sudo -Eu dbbackup stat -c%s /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ filesize=7213
+ write_log notice 'Backup of '\''mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'\'' created with the size of 7213 bytes'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Backup of 'mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz' created with the size of 7213 bytes
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ chmod 600 /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ case "${backup_job_backup_location,,}" in
+ write_log debug 'Moving backup to S3 Bucket'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Moving backup to S3 Bucket
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' -n KEY_ID ']'
+ '[' -n KEY_SECRET ']'
+ export AWS_ACCESS_KEY_ID=KEY_ID
+ AWS_ACCESS_KEY_ID=KEY_ID
+ export AWS_SECRET_ACCESS_KEY=KEY_SECRET
+ AWS_SECRET_ACCESS_KEY=KEY_SECRET
+ export AWS_DEFAULT_REGION=us
+ AWS_DEFAULT_REGION=us
+ '[' -f '' ']'
+ var_true TRUE
+ '[' true = true ']'
+ write_log debug 'Skipping SSL verification for HTTPS S3 Hosts'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:05 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Skipping SSL verification for HTTPS S3 Hosts
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ s3_ssl=--no-verify-ssl
+ [[ -n storage.googleapis.com ]]
+ PARAM_AWS_ENDPOINT_URL=' --endpoint-url https://storage.googleapis.com'
+ silent aws --endpoint-url https://storage.googleapis.com s3 cp /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz s3://fabric-dbbackups-prod/GMFC-RKD/2023/11/21/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz --no-verify-ssl
+ '[' TRUE = true ']'
+ '[' '' = true ']'
+ '[' debug = debug ']'
+ aws --endpoint-url https://storage.googleapis.com s3 cp /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz s3://fabric-dbbackups-prod/GMFC-RKD/2023/11/21/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz --no-verify-ssl
/usr/bin/aws:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').run_script('awscli==1.29.78', 'aws')
/usr/lib/python3.11/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'storage.googleapis.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
upload: ./mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz to s3://fabric-dbbackups-prod/GMFC-RKD/2023/11/21/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ move_exit_code=0
+ '[' MD5 '!=' none ']'
+ silent run_as_user aws --endpoint-url https://storage.googleapis.com s3 cp /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5 s3://fabric-dbbackups-prod/GMFC-RKD/2023/11/21/ --no-verify-ssl
+ '[' TRUE = true ']'
+ '[' '' = true ']'
+ '[' debug = debug ']'
+ run_as_user aws --endpoint-url https://storage.googleapis.com s3 cp /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5 s3://fabric-dbbackups-prod/GMFC-RKD/2023/11/21/ --no-verify-ssl
+ sudo -Eu dbbackup aws --endpoint-url https://storage.googleapis.com s3 cp /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5 s3://fabric-dbbackups-prod/GMFC-RKD/2023/11/21/ --no-verify-ssl
/usr/bin/aws:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').run_script('awscli==1.29.78', 'aws')
/usr/lib/python3.11/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'storage.googleapis.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
upload: ./mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5 to s3://fabric-dbbackups-prod/GMFC-RKD/2023/11/21/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
+ run_as_user rm -rf /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
+ sudo -Eu dbbackup rm -rf /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz.md5
+ run_as_user rm -rf /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ sudo -Eu dbbackup rm -rf /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ run_as_user rm -rf /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ sudo -Eu dbbackup rm -rf /tmp/backups/mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ check_exit_code move mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ write_log debug 'Move exit Code is 0'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:08 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Move exit Code is 0
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${move_exit_code}" in
+ write_log debug 'Moving of backup '\''mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz'\'' completed successfully'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:08 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Moving of backup 'mongo_base_monitor_mongodb-headless_20231121-113005.archive.gz' completed successfully
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ post_dbbackup base_monitor
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
++ run_as_user date +%s
++ sudo -Eu dbbackup date +%s
+ dbbackup_finish_time=1700566208
++ run_as_user echo 3
++ sudo -Eu dbbackup echo 3
+ dbbackup_total_time=3
+ var_true FALSE
+ '[' false = true ']'
+ '[' false = yes ']'
+ '[' -n '' ']'
+ '[' -d /assets/custom-scripts/ ']'
+ '[' -d /assets/scripts/post/ ']'
++ echo 3
++ awk '{printf "Hours: %d Minutes: %02d Seconds: %02d", $1/3600, ($1/60)%60, $1%60}'
+ write_log notice 'DB Backup for '\''base_monitor'\'' time taken: Hours: 0 Minutes: 00 Seconds: 03'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:08 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] DB Backup for 'base_monitor' time taken: Hours: 0 Minutes: 00 Seconds: 03
+ output_on
+ '[' true = true ']'
+ case "$(basename "$0")" in
++ basename ./run
+ case "$PWD" in
+ set -x
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ timer job stop
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ case "${1}" in
+ case "${2}" in
++ date +%s
+ backup_job_finish_time=1700566208
+ backup_job_total_time=3
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' -z '' ']'
+ exitcode_backup=0
++ date -d @1700566208 '+%Y-%m-%d %T %Z'
+ print_info 'Backup 01 routines finish time: 2023-11-21 11:30:08 UTC with exit code 0'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:08 [INFO] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Backup 01 routines finish time: 2023-11-21 11:30:08 UTC with exit code 0
++ echo 3
++ awk '{printf "Hours: %d Minutes: %02d Seconds: %02d", $1/3600, ($1/60)%60, $1%60}'
+ print_notice 'Backup 01 routines time taken: Hours: 0 Minutes: 00 Seconds: 03'
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:08 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Backup 01 routines time taken: Hours: 0 Minutes: 00 Seconds: 03
+ sed -i '/^01/d' /tmp/.container/db-backup-backups
+ symlink_log
+ '[' -n mongo ']'
+ '[' -n base_monitor ']'
+ '[' -n mongodb-headless ']'
+ '[' -n mongo_base_monitor_mongodb-headless ']'
++ pwd
+ local oldpwd=/tmp/backups
++ date +%Y%m%d
+ cd /logs/20231121
++ date +%Y%m%d
++ date -d @1700566205 +%Y%m%d_%H%M%S
+ ln -sf 20231121/20231121_113005-mongo_base_monitor_mongodb-headless.log ../latest-mongo_base_monitor_mongodb-headless.log
+ cd /tmp/backups
+ cleanup_old_data
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' -n '' ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ var_false ''
+ '[' '' = false ']'
+ '[' '' = no ']'
+ var_true ''
+ '[' '' = true ']'
+ '[' '' = yes ']'
+ '[' '!' '' = true ']'
+++ date +%s
++ date -d@1700652605 '+%Y-%m-%d %T %Z'
+ print_notice 'Sleeping for another 86397 seconds. Waking up at 2023-11-22 11:30:05 UTC '
+ output_off
+ '[' true = true ']'
+ set +x
2023-11-21.11:30:08 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-mongodb-headless__base_monitor] Sleeping for another 86397 seconds. Waking up at 2023-11-22 11:30:05 UTC
+ silent sleep 86397
+ '[' TRUE = true ']'
+ '[' '' = true ']'
+ '[' debug = debug ']'
+ sleep 86397
tiredofit commented 10 months ago

Thanks, it looks as if there was a problem with one of the upstream dependencies S6 Overlay: https://github.com/just-containers/s6-overlay/issues/557 causing some abnormal shutdown issues.

Regardless I have made some additional checks and tested and seems to work to what you are trying to do. Please confirm it's working for you now..

4.0.20

ruarkadi commented 10 months ago

Thanks @tiredofit

I was able to get it to work using the new version.

However, I did need to override the default entrypoint of "/init" with the following:

                      containers:
                      - name: {{ include "mongodb.fullname" . }}-mongodump
                        image: "tiredofit/db-backup:4.0.20"
                        command: [
                          "/init",
                          "backup-now"
                        ]
                        env:
                        - name: MODE
                          value: "MANUAL"
                        - name: MANUAL_RUN_FOREVER
                          value: "FALSE"
                        - name: BACKUP_JOB_CONCURRENCY
                          value: "1"
                        - name: CONTAINER_ENABLE_MONITORING
                          value: "FALSE"
                        - name: CONTAINER_ENABLE_SCHEDULING
                          value: "FALSE"
                        - name: CONTAINER_ENABLE_MESSAGING
                          value: "FALSE"
                        - name: CONTAINER_ENABLE_LOGROTATE
                          value: "FALSE"
                        - name: TIMEZONE
                          value: Etc/UTC
                        - name: CONTAINER_NAME
                          value: "{{ include "mongodb.fullname" . }}-mongodump"
                        - name: DEFAULT_LOG_LEVEL
                          value: debug
                        - name: DEBUG_MODE
                          value: "TRUE"                        
                        - name: DEFAULT_COMPRESSION
                          value: GZ
                        - name: DEFAULT_BACKUP_LOCATION
                          value: S3
                        - name: DEFAULT_S3_BUCKET
                          value: test-dbbackups-prod
                        - name: DEFAULT_S3_KEY_ID
                          value: "XXXX"
                        - name: DEFAULT_S3_KEY_SECRET
                          value: "XXXX"
                        - name: DEFAULT_S3_PATH
                          value: {{ .Values.global.clusterName | upper }}/{{ now | date "2006/01/02" }}
                        - name: DEFAULT_S3_REGION
                          value: us
                        - name: DEFAULT_S3_HOST
                          value: storage.googleapis.com
                        - name: DB01_TYPE
                          value: mongo
                        - name: DB01_HOST
                          value: {{ include "mongodb.service.nameOverride" . }}
                        - name: DB01_NAME
                          value: base_monitor
                        - name: DB01_AUTH
                          value: admin
                        - name: DB01_USER
                          value: {{ .Values.auth.rootUser | quote }}
                        - name: DB01_PASS
                          valueFrom:
                            secretKeyRef:
                              name: {{ include "mongodb.secretName" . }}
                              key: mongodb-root-password

Still testing if this would cover all bases. I am wondering whether the MODE switch already has a built-in option of run once and exit.

tiredofit commented 10 months ago

That sounds about right. It was a while ago that this was built for a very similar use case, and I couldn't remember much if MANUAL triggered an actual backup or if it was just putting the container in a standalone state waiting for the job to be triggered by the backup-now commands. I think what you have works and shouldn't cause any issues. There could be some enhancements likely to the mode feature as it probably deserves its own reworking of code as opposed to being lumped in with 3 other variables that do similar things.

ruarkadi commented 10 months ago

Really appreciate the quick turn around. I'm going to implement this as a backup solution for mongo and postgres and see how it goes. This seems like a very good solution configuration wise. It can be templated using helm quite easily. Maybe I'll come back with some suggestions or enhancements down the line.