search

tiredofit / docker-db-backup

Backup multiple database types on a scheduled basis with many customizable options
MIT License
856 stars 125 forks source link

Error blobxfer Error: Got unexpected extra arguments #304

Closed lazarevdmitriy closed 10 months ago

lazarevdmitriy commented 10 months ago

Summary

example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:37 [INFO] ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer
example-mssql-blobxfer-db-backup  | Usage: blobxfer download [OPTIONS]
example-mssql-blobxfer-db-backup  | Try 'blobxfer download -h' for help.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | Error: Got unexpected extra arguments (Add Storage Name}' /backup)

Steps to reproduce

git clone git@github.com:tiredofit/docker-db-backup.git
cd docker-db-backup/examples/mssql-blobxfer
docker-compose up

What is the expected correct behavior?

Backups must be successfully uploaded to the azure blob storage

Relevant logs and/or screenshots

docker-compose up

[+] Building 0.0s (0/0)                                                                              docker:desktop-linux
[+] Running 2/1
 ✔ Container example-mssql-s3-db               Recreated                                                             0.1s
 ✔ Container example-mssql-blobxfer-db-backup  Recreated                                                             0.1s
Attaching to example-mssql-blobxfer-db-backup, example-mssql-s3-db
example-mssql-s3-db               | SQL Server 2019 will run as non-root by default.
example-mssql-s3-db               | This container is running as user mssql.
example-mssql-s3-db               | To learn more visit https://go.microsoft.com/fwlink/?linkid=2099216.
example-mssql-blobxfer-db-backup  | s6-linux-init: warning: parent wrote to fd 3!
example-mssql-blobxfer-db-backup  |                                                                        ,---.
example-mssql-blobxfer-db-backup  | ,--------.,--.                 ,--.            ,---.    ,--. ,--------.|   |
example-mssql-blobxfer-db-backup  | '--.  .--'`--',--.--. ,---.  ,-|  |     ,---. /  .-'    |  | '--.  .--'|  .'
example-mssql-blobxfer-db-backup  |    |  |   ,--.|  .--'| .-. :' .-. |    | .-. ||  `-,    |  |    |  |   |  |
example-mssql-blobxfer-db-backup  |    |  |   |  ||  |   \   --.\ `-' |    ' '-' '|  .-'    |  |.--.|  |   `--'
example-mssql-blobxfer-db-backup  |    `--'   `--'`--'    `----' `---'      `---' `--'      `--''--'`--'   .--.
example-mssql-blobxfer-db-backup  |                                                                        '--'
example-mssql-blobxfer-db-backup  | Image:  tiredofit/db-backup
example-mssql-blobxfer-db-backup  | Repository/Documentation: https://github.com/tiredofit/docker-db-backup/
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | If this image provides you value  - Consider sponsoring my work for continued
example-mssql-blobxfer-db-backup  | development, timely updates, and feature requests. Commercial support available.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  |                     More Info:  https://www.tiredofit.ca
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | 2023-12-04.15:51:26 [NOTICE] ** [timezone] Timezone: Setting to 'America/Vancouver' from 'Etc/GMT'
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:27 [NOTICE] ** [scheduling] Container configured for scheduled tasks with 'cron'
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:28 [NOTICE] ** [messaging] Container configured to route mail via SMTP to 'postfix-relay'
example-mssql-s3-db               | 2023-12-04 15:51:30.20 Server      The licensing PID was successfully processed. The new edition is [Express Edition].
2023-12-04 15:51:30.48 Server      Setup step is copying system data file 'C:\templatedata\master.mdf' to '/var/opt/mssql/data/master.mdf'.
2023-12-04 15:51:30.49 Server      Did not find an existing master data file /var/opt/mssql/data/master.mdf, copying the missing default master and other system database files. If you have moved the database location, but not moved the database files, startup may fail. To repair: shutdown SQL Server, move the master database to configured location, and restart.
2023-12-04 15:51:30.49 Server      Setup step is copying system data file 'C:\templatedata\mastlog.ldf' to '/var/opt/mssql/data/mastlog.ldf'.
2023-12-04 15:51:30.50 Server      Setup step is copying system data file 'C:\templatedata\model.mdf' to '/var/opt/mssql/data/model.mdf'.
2023-12-04 15:51:30.51 Server      Setup step is copying system data file 'C:\templatedata\modellog.ldf' to '/var/opt/mssql/data/modellog.ldf'.
2023-12-04 15:51:30.52 Server      Setup step is copying system data file 'C:\templatedata\msdbdata.mdf' to '/var/opt/mssql/data/msdbdata.mdf'.
2023-12-04 15:51:30.54 Server      Setup step is copying system data file 'C:\templatedata\msdblog.ldf' to '/var/opt/mssql/data/msdblog.ldf'.
2023-12-04 15:51:30.55 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.mdf' to '/var/opt/mssql/data/model_replicatedmaster.mdf'.
2023-12-04 15:51:30.56 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.ldf' to '/var/opt/mssql/data/model_replicatedmaster.ldf'.
2023-12-04 15:51:30.56 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdbdata.mdf' to '/var/opt/mssql/data/model_msdbdata.mdf'.
2023-12-04 15:51:30.58 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdblog.ldf' to '/var/opt/mssql/data/model_msdblog.ldf'.
2023-12-04 15:51:30.61 Server      Microsoft SQL Server 2019 (RTM-CU23) (KB5030333) - 15.0.4335.1 (X64)
example-Sep 21 2023 17:28:44      |
example-Copyright (C) 2019 Microsoft Corporation
example-Express Edition (64-bit) on Linux (Ubuntu 20.04.6 LTS) <X64>
2023-12-04 15:51:30.62 Server      UTC adjustment: 0:00
2023-12-04 15:51:30.62 Server      (c) Microsoft Corporation.
2023-12-04 15:51:30.62 Server      All rights reserved.
2023-12-04 15:51:30.62 Server      Server process ID is 380.
2023-12-04 15:51:30.62 Server      Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2023-12-04 15:51:30.62 Server      Registry startup parameters:
example- -d /var/opt/mssql/data/master.mdf
example- -l /var/opt/mssql/data/mastlog.ldf
example- -e /var/opt/mssql/log/errorlog
2023-12-04 15:51:30.63 Server      SQL Server detected 1 sockets with 8 cores per socket and 8 logical processors per socket, 8 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required.
2023-12-04 15:51:30.63 Server      SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.
2023-12-04 15:51:30.63 Server      Detected 3141 MB of RAM. This is an informational message; no user action is required.
2023-12-04 15:51:30.63 Server      Using conventional memory in the memory manager.
2023-12-04 15:51:30.63 Server      Page exclusion bitmap is enabled.
2023-12-04 15:51:30.66 Server      Buffer pool extension is not supported on Linux platform.
2023-12-04 15:51:30.66 Server      Buffer Pool: Allocating 524288 bytes for 403167 hashPages.
2023-12-04 15:51:30.77 Server      Buffer pool extension is already disabled. No action is necessary.
2023-12-04 15:51:31.01 Server      Successfully initialized the TLS configuration. Allowed TLS protocol versions are ['1.0 1.1 1.2']. Allowed TLS ciphers are ['ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!DHE-RSA-AES256-GCM-SHA384:!DHE-RSA-AES128-GCM-SHA256:!DHE-RSA-AES256-SHA:!DHE-RSA-AES128-SHA'].
2023-12-04 15:51:31.07 Server      Query Store settings initialized with enabled = 1,
SetThreadAffinity failed: 87      |
2023-12-04 15:51:31.11 Server      The maximum number of dedicated administrator connections for this instance is '1'
2023-12-04 15:51:31.11 Server      Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2023-12-04 15:51:31.12 Server      Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This is an informational message only.  No user action is required.
2023-12-04 15:51:31.15 Server      In-Memory OLTP initialized on lowend machine.
2023-12-04 15:51:31.26 Server      [INFO] Created Extended Events session 'hkenginexesession'
2023-12-04 15:51:31.27 Server      Database Instant File Initialization: enabled. For security and performance considerations see the topic 'Database Instant File Initialization' in SQL Server Books Online. This is an informational message only. No user action is required.
ForceFlush is enabled for this instance.
2023-12-04 15:51:31.28 Server      Total Log Writer threads: 2. This is an informational message; no user action is required.
2023-12-04 15:51:31.28 Server      Software Usage Metrics is disabled.
2023-12-04 15:51:31.32 spid9s      [1]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 15:51:31.33 spid9s      Starting up database 'master'.
2023-12-04 15:51:31.37 Server      CLR version v4.0.30319 loaded.
ForceFlush feature is enabled for log durability.
2023-12-04 15:51:31.49 spid9s      The tail of the log for database master is being rewritten to match the new sector size of 4096 bytes.  1536 bytes at offset 412160 in file /var/opt/mssql/data/mastlog.ldf will be written.
2023-12-04 15:51:31.68 spid9s      Converting database 'master' from version 897 to the current version 904.
2023-12-04 15:51:31.68 spid9s      Database 'master' running the upgrade step from version 897 to version 898.
2023-12-04 15:51:31.72 spid9s      Database 'master' running the upgrade step from version 898 to version 899.
2023-12-04 15:51:31.98 spid9s      Database 'master' running the upgrade step from version 899 to version 900.
2023-12-04 15:51:32.05 spid9s      Database 'master' running the upgrade step from version 900 to version 901.
2023-12-04 15:51:32.07 spid9s      Database 'master' running the upgrade step from version 901 to version 902.
2023-12-04 15:51:32.11 spid9s      Database 'master' running the upgrade step from version 902 to version 903.
2023-12-04 15:51:32.13 spid9s      Database 'master' running the upgrade step from version 903 to version 904.
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:32 [STARTING] ** [scheduling] [1] Starting cron
2023-12-04 15:51:32.43 spid9s      SQL Server Audit is starting the audits. This is an informational message. No user action is required.
2023-12-04 15:51:32.44 spid9s      SQL Server Audit has started the audits. This is an informational message. No user action is required.
2023-12-04 15:51:32.48 spid9s      SQL Trace ID 1 was started by login "sa".
2023-12-04 15:51:32.50 spid9s      Server name is 'example-db-host'. This is an informational message only. No user action is required.
2023-12-04 15:51:32.50 spid23s     Password policy update was successful.
2023-12-04 15:51:32.52 spid24s     Always On: The availability replica manager is starting. This is an informational message only. No user action is required.
2023-12-04 15:51:32.53 spid24s     Always On: The availability replica manager is waiting for the instance of SQL Server to allow client connections. This is an informational message only. No user action is required.
2023-12-04 15:51:32.53 spid9s      [4]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 15:51:32.53 spid12s     [32767]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 15:51:32.55 spid9s      Starting up database 'msdb'.
2023-12-04 15:51:32.55 spid12s     Starting up database 'mssqlsystemresource'.
2023-12-04 15:51:32.56 spid12s     The resource database build version is 15.00.4335. This is an informational message only. No user action is required.
2023-12-04 15:51:32.59 spid12s     [3]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 15:51:32.59 spid12s     Starting up database 'model'.
2023-12-04 15:51:32.62 spid9s      Converting database 'msdb' from version 897 to the current version 904.
2023-12-04 15:51:32.62 spid9s      Database 'msdb' running the upgrade step from version 897 to version 898.
2023-12-04 15:51:32.62 spid12s     The tail of the log for database model is being rewritten to match the new sector size of 4096 bytes.  1536 bytes at offset 76288 in file /var/opt/mssql/data/modellog.ldf will be written.
2023-12-04 15:51:32.66 spid9s      Database 'msdb' running the upgrade step from version 898 to version 899.
2023-12-04 15:51:32.68 spid12s     Converting database 'model' from version 897 to the current version 904.
2023-12-04 15:51:32.68 spid12s     Database 'model' running the upgrade step from version 897 to version 898.
2023-12-04 15:51:32.71 spid12s     Database 'model' running the upgrade step from version 898 to version 899.
2023-12-04 15:51:32.71 spid23s     A self-generated certificate was successfully loaded for encryption.
2023-12-04 15:51:32.72 spid9s      Database 'msdb' running the upgrade step from version 899 to version 900.
2023-12-04 15:51:32.74 spid23s     Server is listening on [ 'any' <ipv6> 1433].
2023-12-04 15:51:32.75 spid23s     Server is listening on [ 'any' <ipv4> 1433].
2023-12-04 15:51:32.75 spid9s      Database 'msdb' running the upgrade step from version 900 to version 901.
2023-12-04 15:51:32.75 spid23s     Dedicated administrator connection support was not started because it is disabled on this edition of SQL Server. If you want to use a dedicated administrator connection, restart SQL Server using the trace flag 7806. This is an informational message only. No user action is required.
2023-12-04 15:51:32.76 spid12s     Database 'model' running the upgrade step from version 899 to version 900.
2023-12-04 15:51:32.76 spid9s      Database 'msdb' running the upgrade step from version 901 to version 902.
2023-12-04 15:51:32.77 spid23s     Server is listening on [ ::1 <ipv6> 1431].
2023-12-04 15:51:32.77 spid23s     Server is listening on [ 127.0.0.1 <ipv4> 1431].
2023-12-04 15:51:32.77 spid23s     SQL Server is now ready for client connections. This is an informational message; no user action is required.
2023-12-04 15:51:32.78 spid12s     Database 'model' running the upgrade step from version 900 to version 901.
2023-12-04 15:51:32.78 spid12s     Database 'model' running the upgrade step from version 901 to version 902.
2023-12-04 15:51:32.79 spid12s     Database 'model' running the upgrade step from version 902 to version 903.
2023-12-04 15:51:32.79 spid12s     Database 'model' running the upgrade step from version 903 to version 904.
2023-12-04 15:51:32.83 Server      Common language runtime (CLR) functionality initialized.
2023-12-04 15:51:32.91 spid12s     Clearing tempdb database.
2023-12-04 15:51:32.97 spid12s     [2]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 15:51:32.97 spid12s     Starting up database 'tempdb'.
2023-12-04 15:51:33.15 spid24s     The Service Broker endpoint is in disabled or stopped state.
2023-12-04 15:51:33.15 spid24s     The Database Mirroring endpoint is in disabled or stopped state.
2023-12-04 15:51:33.18 spid24s     Service Broker manager has started.
2023-12-04 15:51:33.24 spid9s      Database 'msdb' running the upgrade step from version 902 to version 903.
2023-12-04 15:51:33.24 spid9s      Database 'msdb' running the upgrade step from version 903 to version 904.
2023-12-04 15:51:33.38 spid9s      Recovery is complete. This is an informational message only. No user action is required.
2023-12-04 15:51:33.47 spid18s     The default language (LCID 0) has been set for engine and full-text services.
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:36 [INFO] ** [01-example-db-host__test1] Next Backup at 2023-12-04 07:51:36 PST
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:36 [NOTICE] ** [01-example-db-host__test1] Dumping MSSQL database: ''
2023-12-04 15:51:36.91 Backup      Error: 3041, Severity: 16, State: 1.
2023-12-04 15:51:36.91 Backup      BACKUP failed to complete the command BACKUP DATABASE test1. Check the backup application log for detailed messages.
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:37 [INFO] ** [01-example-db-host__test1] DB Backup of 'mssql_test1_example-db-host_20231204-075136.bak.gz' completed successfully
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:37 [NOTICE] ** [01-example-db-host__test1] Generating SHA1 sum for 'mssql_test1_example-db-host_20231204-075136.bak.gz'
example-mssql-blobxfer-db-backup  | sha1sum: mssql_test1_example-db-host_20231204-075136.bak.gz: No such file or directory
example-mssql-blobxfer-db-backup  | stat: cannot stat '/tmp/backups/mssql_test1_example-db-host_20231204-075136.bak.gz': No such file or directory
example-mssql-blobxfer-db-backup  | date: /tmp/backups/mssql_test1_example-db-host_20231204-075136.bak.gz: No such file or directory
example-mssql-blobxfer-db-backup  | stat: cannot stat '/tmp/backups/mssql_test1_example-db-host_20231204-075136.bak.gz': No such file or directory
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:37 [NOTICE] ** [01-example-db-host__test1] Backup of 'mssql_test1_example-db-host_20231204-075136.bak.gz' created with the size of  bytes
example-mssql-blobxfer-db-backup  | chmod: cannot access '/tmp/backups/mssql_test1_example-db-host_20231204-075136.bak.gz': No such file or directory
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:37 [INFO] ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer
example-mssql-blobxfer-db-backup  | Usage: blobxfer download [OPTIONS]
example-mssql-blobxfer-db-backup  | Try 'blobxfer download -h' for help.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | Error: Got unexpected extra arguments (Add Storage Name}' /backup)
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:38 [INFO] ** [01-example-db-host__test1] Moving backup to external storage with blobxfer
example-mssql-blobxfer-db-backup  | mv: cannot stat '/tmp/backups/mssql_test1_example-db-host_20231204-075136.bak.gz': No such file or directory
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:39 [ERROR] ** [01-example-db-host__test1] Moving of backup 'mssql_test1_example-db-host_20231204-075136.bak.gz' reported errors
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:39 [NOTICE] ** [01-example-db-host__test1] DB Backup for 'test1' time taken: Hours: 0 Minutes: 00 Seconds: 03
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:39 [ERROR] ** [01-example-db-host__test1] Skipping Cleaning up old backups because there were errors in backing up
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:39 [INFO] ** [01-example-db-host__test1] Backup 01 routines finish time: 2023-12-04 07:51:39 PST with exit code 0
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:39 [NOTICE] ** [01-example-db-host__test1] Backup 01 routines time taken: Hours: 0 Minutes: 00 Seconds: 03
example-mssql-blobxfer-db-backup  | 2023-12-04.07:51:39 [NOTICE] ** [01-example-db-host__test1] Sleeping for another 86397 seconds. Waking up at 2023-12-05 07:51:36 PST

Environment


#
# Example for Microsoft SQL Server
# upload with blobxfer to azure storage
#

services:
  example-mssql-s3-db:
    hostname: example-db-host
    image: mcr.microsoft.com/mssql/server:2019-latest
    platform: linux/amd64
    container_name: example-mssql-s3-db
    restart: unless-stopped
    ports:
      - "127.0.0.1:11433:1433"
    networks:
      example-mssql-blobxfer-net:
    volumes:
      - ./tmp/backups:/tmp/backups    # shared tmp backup directory
    environment:
      ACCEPT_EULA: Y
      MSSQL_SA_PASSWORD: 5hQa0utRFBpIY3yhoIyE
      MSSQL_PID: Express

  example-mssql-blobxfer-db-backup:
    container_name: example-mssql-blobxfer-db-backup
    # if you want to build and use image from current source
    # execute in terminal --> docker build -t tiredofit/db-backup-mssql-blobxfer .
    # replace --> image: tiredofit/db-backup-mssql
    # image: tiredofit/db-backup
    image: tiredofit/db-backup
    platform: linux/amd64
    links:
     - example-mssql-s3-db
    volumes:
      - ./backups:/backup
      - ./tmp/backups:/tmp/backups    # shared tmp backup directory
      #- ./post-script.sh:/assets/custom-scripts/post-script.sh
    environment:
      - TIMEZONE=America/Vancouver
      - CONTAINER_ENABLE_MONITORING=FALSE
      - CONTAINER_NAME=example-mssql-blobxfer-db-backup
      # - DEBUG_MODE=TRUE
      - DB01_TYPE=mssql
      - DB01_HOST=example-db-host
      # - DB01_PORT=1488
      # create database with name `test1` manually first
      - DB01_NAME=test1             # Create this database
      - DB01_USER=sa
      - DB01_PASS=5hQa0utRFBpIY3yhoIyE
      - DB01_DUMP_INTERVAL=5        # backup every 5 minute
      # - DB01_DUMP_BEGIN=0000      # backup starts at midnight vs not set immediately
      - DB01_CLEANUP_TIME=60         # clean backups they are older than 60 minutes
      - DB01_CHECKSUM=SHA1          # Set Checksum to be SHA1
      - DB01_COMPRESSION=GZ         # Set compression to use GZIP

      # === S3 Blobxfer ===
      - DB01_BACKUP_LOCATION=blobxfer
      # Add here azure storage account
      - DB01_BLOBXFER_STORAGE_ACCOUNT="account"
      # Add here azure storage account key
      - SB01_BLOBXFER_STORAGE_ACCOUNT_KEY="key"
      - DB01_BLOBXFER_REMOTE_PATH=docker-db-backup
    restart: always
    networks:
      example-mssql-blobxfer-net:

networks:
  example-mssql-blobxfer-net:
    name: example-mssql-blobxfer-net
tiredofit commented 10 months ago

Can you pull again, and then run with DEBUG_BACKUP_MSSQL=TRUE ? This very much sounds similar to #299 however in your case your backup is not completing but is returning that it is successful:

2023-12-04 15:51:36.91 Backup      Error: 3041, Severity: 16, State: 1.
2023-12-04 15:51:36.91 Backup      BACKUP failed to complete the command BACKUP DATABASE test1. Check the backup application log for detailed messages.
lazarevdmitriy commented 10 months ago

I deleted everything and copied the repository again. Launched docker-compose up, connected to ms sql using dbeaver and created the test1 database. Restarted docker-compose. The backup was successful, but the copy failed.

docker-compose up

[+] Building 0.0s (0/0)                                                                              docker:desktop-linux
[+] Running 2/0
 ✔ Container example-mssql-s3-db               Created                                                               0.0s
 ✔ Container example-mssql-blobxfer-db-backup  Created                                                               0.0s
Attaching to example-mssql-blobxfer-db-backup, example-mssql-s3-db
example-mssql-s3-db               | SQL Server 2019 will run as non-root by default.
example-mssql-s3-db               | This container is running as user mssql.
example-mssql-s3-db               | Your master database file is owned by mssql.
example-mssql-s3-db               | To learn more visit https://go.microsoft.com/fwlink/?linkid=2099216.
example-mssql-blobxfer-db-backup  | s6-linux-init: warning: parent wrote to fd 3!
example-mssql-blobxfer-db-backup  |                                                                        ,---.
example-mssql-blobxfer-db-backup  | ,--------.,--.                 ,--.            ,---.    ,--. ,--------.|   |
example-mssql-blobxfer-db-backup  | '--.  .--'`--',--.--. ,---.  ,-|  |     ,---. /  .-'    |  | '--.  .--'|  .'
example-mssql-blobxfer-db-backup  |    |  |   ,--.|  .--'| .-. :' .-. |    | .-. ||  `-,    |  |    |  |   |  |
example-mssql-blobxfer-db-backup  |    |  |   |  ||  |   \   --.\ `-' |    ' '-' '|  .-'    |  |.--.|  |   `--'
example-mssql-blobxfer-db-backup  |    `--'   `--'`--'    `----' `---'      `---' `--'      `--''--'`--'   .--.
example-mssql-blobxfer-db-backup  |                                                                        '--'
example-mssql-blobxfer-db-backup  | Image:  tiredofit/db-backup | Version  4.0.26 Type 'image_changelog' for details
example-mssql-blobxfer-db-backup  | Repository/Documentation: https://github.com/tiredofit/docker-db-backup/
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | If this image provides you value  - Consider sponsoring my work for continued
example-mssql-blobxfer-db-backup  | development, timely updates, and feature requests. Commercial support available.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  |                     More Info:  https://www.tiredofit.ca
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:33 [NOTICE] ** [container-init] Detected Container that has been restarted - Cleaning '/tmp/.container' files
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:35 [NOTICE] ** [scheduling] Container configured for scheduled tasks with 'cron'
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:36 [NOTICE] ** [messaging] Container configured to route mail via SMTP to 'postfix-relay'
example-mssql-s3-db               | 2023-12-04 16:54:37.31 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.mdf' to '/var/opt/mssql/data/model_replicatedmaster.mdf'.
2023-12-04 16:54:37.35 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.ldf' to '/var/opt/mssql/data/model_replicatedmaster.ldf'.
2023-12-04 16:54:37.35 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdbdata.mdf' to '/var/opt/mssql/data/model_msdbdata.mdf'.
2023-12-04 16:54:37.37 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdblog.ldf' to '/var/opt/mssql/data/model_msdblog.ldf'.
2023-12-04 16:54:37.42 Server      Microsoft SQL Server 2019 (RTM-CU23) (KB5030333) - 15.0.4335.1 (X64)
example-Sep 21 2023 17:28:44      |
example-Copyright (C) 2019 Microsoft Corporation
example-Express Edition (64-bit) on Linux (Ubuntu 20.04.6 LTS) <X64>
2023-12-04 16:54:37.42 Server      UTC adjustment: 0:00
2023-12-04 16:54:37.42 Server      (c) Microsoft Corporation.
2023-12-04 16:54:37.42 Server      All rights reserved.
2023-12-04 16:54:37.42 Server      Server process ID is 384.
2023-12-04 16:54:37.42 Server      Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2023-12-04 16:54:37.42 Server      Registry startup parameters:
example- -d /var/opt/mssql/data/master.mdf
example- -l /var/opt/mssql/data/mastlog.ldf
example- -e /var/opt/mssql/log/errorlog
2023-12-04 16:54:37.43 Server      SQL Server detected 1 sockets with 8 cores per socket and 8 logical processors per socket, 8 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required.
2023-12-04 16:54:37.43 Server      SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.
2023-12-04 16:54:37.43 Server      Detected 3141 MB of RAM. This is an informational message; no user action is required.
2023-12-04 16:54:37.43 Server      Using conventional memory in the memory manager.
2023-12-04 16:54:37.44 Server      Page exclusion bitmap is enabled.
2023-12-04 16:54:37.46 Server      Buffer pool extension is not supported on Linux platform.
2023-12-04 16:54:37.46 Server      Buffer Pool: Allocating 524288 bytes for 394367 hashPages.
2023-12-04 16:54:37.57 Server      Buffer pool extension is already disabled. No action is necessary.
2023-12-04 16:54:37.77 Server      Successfully initialized the TLS configuration. Allowed TLS protocol versions are ['1.0 1.1 1.2']. Allowed TLS ciphers are ['ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!DHE-RSA-AES256-GCM-SHA384:!DHE-RSA-AES128-GCM-SHA256:!DHE-RSA-AES256-SHA:!DHE-RSA-AES128-SHA'].
2023-12-04 16:54:37.83 Server      Query Store settings initialized with enabled = 1,
SetThreadAffinity failed: 87      |
2023-12-04 16:54:37.86 Server      The maximum number of dedicated administrator connections for this instance is '1'
2023-12-04 16:54:37.86 Server      Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2023-12-04 16:54:37.87 Server      Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This is an informational message only.  No user action is required.
2023-12-04 16:54:37.90 Server      In-Memory OLTP initialized on lowend machine.
2023-12-04 16:54:37.97 Server      [INFO] Created Extended Events session 'hkenginexesession'
2023-12-04 16:54:37.97 Server      Database Instant File Initialization: enabled. For security and performance considerations see the topic 'Database Instant File Initialization' in SQL Server Books Online. This is an informational message only. No user action is required.
ForceFlush is enabled for this instance.
2023-12-04 16:54:37.98 Server      Total Log Writer threads: 2. This is an informational message; no user action is required.
2023-12-04 16:54:38.02 Server      Software Usage Metrics is disabled.
2023-12-04 16:54:38.04 spid9s      [1]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 16:54:38.05 spid9s      Starting up database 'master'.
2023-12-04 16:54:38.10 Server      CLR version v4.0.30319 loaded.
ForceFlush feature is enabled for log durability.
2023-12-04 16:54:38.43 spid9s      SQL Server Audit is starting the audits. This is an informational message. No user action is required.
2023-12-04 16:54:38.43 spid9s      SQL Server Audit has started the audits. This is an informational message. No user action is required.
2023-12-04 16:54:38.55 spid9s      SQL Trace ID 1 was started by login "sa".
2023-12-04 16:54:38.62 spid9s      Server name is 'example-db-host'. This is an informational message only. No user action is required.
2023-12-04 16:54:38.65 spid23s     Always On: The availability replica manager is starting. This is an informational message only. No user action is required.
2023-12-04 16:54:38.66 spid23s     Always On: The availability replica manager is waiting for the instance of SQL Server to allow client connections. This is an informational message only. No user action is required.
2023-12-04 16:54:38.68 spid9s      [4]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 16:54:38.69 spid9s      Starting up database 'msdb'.
2023-12-04 16:54:38.69 spid12s     [32767]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 16:54:38.70 spid12s     Starting up database 'mssqlsystemresource'.
2023-12-04 16:54:38.72 spid12s     The resource database build version is 15.00.4335. This is an informational message only. No user action is required.
2023-12-04 16:54:38.78 spid12s     [3]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 16:54:38.78 spid12s     Starting up database 'model'.
2023-12-04 16:54:38.84 spid22s     A self-generated certificate was successfully loaded for encryption.
2023-12-04 16:54:38.89 spid22s     Server is listening on [ 'any' <ipv6> 1433].
2023-12-04 16:54:38.90 spid22s     Server is listening on [ 'any' <ipv4> 1433].
2023-12-04 16:54:38.91 spid22s     Dedicated administrator connection support was not started because it is disabled on this edition of SQL Server. If you want to use a dedicated administrator connection, restart SQL Server using the trace flag 7806. This is an informational message only. No user action is required.
2023-12-04 16:54:38.94 spid12s     Clearing tempdb database.
2023-12-04 16:54:38.94 spid22s     Server is listening on [ ::1 <ipv6> 1431].
2023-12-04 16:54:38.95 spid22s     Server is listening on [ 127.0.0.1 <ipv4> 1431].
2023-12-04 16:54:38.95 spid22s     SQL Server is now ready for client connections. This is an informational message; no user action is required.
2023-12-04 16:54:39.12 spid12s     [2]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 16:54:39.12 spid12s     Starting up database 'tempdb'.
2023-12-04 16:54:39.33 spid23s     The Service Broker endpoint is in disabled or stopped state.
2023-12-04 16:54:39.33 spid23s     The Database Mirroring endpoint is in disabled or stopped state.
2023-12-04 16:54:39.47 spid23s     Service Broker manager has started.
2023-12-04 16:54:39.48 spid9s      Recovery is complete. This is an informational message only. No user action is required.
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:39 [STARTING] ** [scheduling] [1] Starting cron
2023-12-04 16:54:40.33 Server      Common language runtime (CLR) functionality initialized.
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:44 [INFO] ** [01-example-db-host__test1] Next Backup at 2023-12-04 08:54:44 PST
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:44 [NOTICE] ** [01-example-db-host__test1] Dumping MSSQL database: ''
example-mssql-blobxfer-db-backup  | + silent run_as_user /opt/mssql-tools18/bin/sqlcmd -C -S example-db-host,1433 -U sa -P 5hQa0utRFBpIY3yhoIyE -Q 'BACKUP DATABASE [test1] TO DISK = N'\''/tmp/backups/mssql_test1_example-db-host_20231204-085444.bak'\'' WITH NOFORMAT, NOINIT, NAME = '\''test1-full'\'', SKIP, NOREWIND, NOUNLOAD, STATS = 10'
example-mssql-blobxfer-db-backup  | + '[' FALSE = true ']'
example-mssql-blobxfer-db-backup  | + '[' true = true ']'
example-mssql-blobxfer-db-backup  | + run_as_user /opt/mssql-tools18/bin/sqlcmd -C -S example-db-host,1433 -U sa -P 5hQa0utRFBpIY3yhoIyE -Q 'BACKUP DATABASE [test1] TO DISK = N'\''/tmp/backups/mssql_test1_example-db-host_20231204-085444.bak'\'' WITH NOFORMAT, NOINIT, NAME = '\''test1-full'\'', SKIP, NOREWIND, NOUNLOAD, STATS = 10'
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup /opt/mssql-tools18/bin/sqlcmd -C -S example-db-host,1433 -U sa -P 5hQa0utRFBpIY3yhoIyE -Q 'BACKUP DATABASE [test1] TO DISK = N'\''/tmp/backups/mssql_test1_example-db-host_20231204-085444.bak'\'' WITH NOFORMAT, NOINIT, NAME = '\''test1-full'\'', SKIP, NOREWIND, NOUNLOAD, STATS = 10'
2023-12-04 16:54:44.61 spid51      [5]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 16:54:44.61 spid51      Starting up database 'test1'.
2023-12-04 16:54:44.65 spid51      Parallel redo is started for database 'test1' with worker pool size [2].
2023-12-04 16:54:44.66 spid51      Parallel redo is shutdown for database 'test1' with worker pool size [2].
2023-12-04 16:54:44.76 Backup      Database backed up. Database: test1, creation date(time): 2023/12/04(16:54:06), pages dumped: 339, first LSN: 36:320:1, last LSN: 36:344:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {'/tmp/backups/mssql_test1_example-db-host_20231204-085444.bak'}). This is an informational message only. No user action is required.
2023-12-04 16:54:44.86 Backup      BACKUP DATABASE successfully processed 330 pages in 0.043 seconds (59.865 MB/sec).
example-mssql-blobxfer-db-backup  | 10 percent processed.
example-mssql-blobxfer-db-backup  | 20 percent processed.
example-mssql-blobxfer-db-backup  | 30 percent processed.
example-mssql-blobxfer-db-backup  | 40 percent processed.
example-mssql-blobxfer-db-backup  | 50 percent processed.
example-mssql-blobxfer-db-backup  | 61 percent processed.
example-mssql-blobxfer-db-backup  | 71 percent processed.
example-mssql-blobxfer-db-backup  | 81 percent processed.
example-mssql-blobxfer-db-backup  | 91 percent processed.
example-mssql-blobxfer-db-backup  | 100 percent processed.
example-mssql-blobxfer-db-backup  | Processed 328 pages for database 'test1', file 'test1' on file 1.
example-mssql-blobxfer-db-backup  | Processed 2 pages for database 'test1', file 'test1_log' on file 1.
example-mssql-blobxfer-db-backup  | BACKUP DATABASE successfully processed 330 pages in 0.043 seconds (59.865 MB/sec).
example-mssql-blobxfer-db-backup  | + exit_code=0
example-mssql-blobxfer-db-backup  | + var_true TRUE
example-mssql-blobxfer-db-backup  | + '[' true = true ']'
example-mssql-blobxfer-db-backup  | + debug off
example-mssql-blobxfer-db-backup  | + case "${1}" in
example-mssql-blobxfer-db-backup  | + backup_job_log_level='notice}'
example-mssql-blobxfer-db-backup  | + CONTAINER_LOG_LEVEL=NOTICE
example-mssql-blobxfer-db-backup  | + DEBUG_MODE=FALSE
example-mssql-blobxfer-db-backup  | + SHOW_OUTPUT=
example-mssql-blobxfer-db-backup  | + var_true FALSE
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = yes ']'
example-mssql-blobxfer-db-backup  | + set +x
example-mssql-blobxfer-db-backup  | + run_as_user pigz -q -3 -p 8 /tmp/backups/mssql_test1_example-db-host_20231204-085444.bak
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup pigz -q -3 -p 8 /tmp/backups/mssql_test1_example-db-host_20231204-085444.bak
example-mssql-blobxfer-db-backup  | + check_exit_code backup mssql_test1_example-db-host_20231204-085444.bak.gz
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + case "${1}" in
example-mssql-blobxfer-db-backup  | + write_log debug 'DB Backup exit Code is 0'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=debug
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=DB Backup exit Code is 0'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 08:54:44 PST [debug] DB Backup exit Code is 0'
example-mssql-blobxfer-db-backup  | ++ date -d @1701708884 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_085444-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_085444-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_debug 'DB Backup exit Code is 0'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + case "${CONTAINER_LOG_LEVEL,,}" in
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.08:54:45 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.08:54:45 \e[45m[DEBUG]\e[49m ** [01-example-db-host__test1] DB Backup exit Code is 0'
example-mssql-blobxfer-db-backup  | + case "${CONTAINER_LOG_FILE_LEVEL,,}" in
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:45 [DEBUG] ** [01-example-db-host__test1] DB Backup exit Code is 0
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.08:54:45 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.08:54:45 [DEBUG] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] DB Backup exit Code is 0'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + case "${exit_code}" in
example-mssql-blobxfer-db-backup  | + write_log info 'DB Backup of '\''mssql_test1_example-db-host_20231204-085444.bak.gz'\'' completed successfully'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=info
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=DB Backup of '\''mssql_test1_example-db-host_20231204-085444.bak.gz'\'' completed successfully'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 08:54:45 PST [info] DB Backup of '\''mssql_test1_example-db-host_20231204-085444.bak.gz'\'' completed successfully'
example-mssql-blobxfer-db-backup  | ++ date -d @1701708884 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_085444-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_085444-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_info 'DB Backup of '\''mssql_test1_example-db-host_20231204-085444.bak.gz'\'' completed successfully'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.08:54:45 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.08:54:45 \e[42m[INFO]\e[49m ** [01-example-db-host__test1] DB Backup of '\''mssql_test1_example-db-host_20231204-085444.bak.gz'\'' completed successfully'
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:45 [INFO] ** [01-example-db-host__test1] DB Backup of 'mssql_test1_example-db-host_20231204-085444.bak.gz' completed successfully
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.08:54:45 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.08:54:45 [INFO] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] DB Backup of '\''mssql_test1_example-db-host_20231204-085444.bak.gz'\'' completed successfully'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + var_true TRUE
example-mssql-blobxfer-db-backup  | + '[' true = true ']'
example-mssql-blobxfer-db-backup  | + debug off
example-mssql-blobxfer-db-backup  | + case "${1}" in
example-mssql-blobxfer-db-backup  | + backup_job_log_level='notice}'
example-mssql-blobxfer-db-backup  | + CONTAINER_LOG_LEVEL=NOTICE
example-mssql-blobxfer-db-backup  | + DEBUG_MODE=FALSE
example-mssql-blobxfer-db-backup  | + SHOW_OUTPUT=
example-mssql-blobxfer-db-backup  | + var_true FALSE
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = yes ']'
example-mssql-blobxfer-db-backup  | + set +x
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:45 [NOTICE] ** [01-example-db-host__test1] Generating SHA1 sum for 'mssql_test1_example-db-host_20231204-085444.bak.gz'
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:45 [NOTICE] ** [01-example-db-host__test1] Backup of 'mssql_test1_example-db-host_20231204-085444.bak.gz' created with the size of 369348 bytes
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:45 [INFO] ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer
example-mssql-blobxfer-db-backup  | Usage: blobxfer download [OPTIONS]
example-mssql-blobxfer-db-backup  | Try 'blobxfer download -h' for help.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | Error: Got unexpected extra argument (/backup)
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:46 [INFO] ** [01-example-db-host__test1] Moving backup to external storage with blobxfer
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:46 [ERROR] ** [01-example-db-host__test1] Moving of backup 'mssql_test1_example-db-host_20231204-085444.bak.gz' reported errors
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:47 [NOTICE] ** [01-example-db-host__test1] DB Backup for 'test1' time taken: Hours: 0 Minutes: 00 Seconds: 02
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:47 [ERROR] ** [01-example-db-host__test1] Skipping Cleaning up old backups because there were errors in backing up
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:47 [INFO] ** [01-example-db-host__test1] Backup 01 routines finish time: 2023-12-04 08:54:47 PST with exit code 0
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:47 [NOTICE] ** [01-example-db-host__test1] Backup 01 routines time taken: Hours: 0 Minutes: 00 Seconds: 03
example-mssql-blobxfer-db-backup  | 2023-12-04.08:54:47 [NOTICE] ** [01-example-db-host__test1] Sleeping for another 86397 seconds. Waking up at 2023-12-05 08:54:44 PST
lazarevdmitriy commented 10 months ago

It seems like something is wrong with blobxfer arguments

tiredofit commented 10 months ago

This sounds similar to the compression issue above..

If you still have the container up can you do this:

ls -l /tmp/backups/*
sudo -Eu dbbackup pigz -q -3 -p 8 /tmp/backups/mssql_test1_example-db-host_20231204-085444.bak
ls -l /tmp/backups/*
lazarevdmitriy commented 10 months ago 
docker exec -it example-mssql-blobxfer-db-backup bash
[tiredofit/db-backup:4.0.27 09:32:46 /] $ ls -l /tmp/backups/*
ls: cannot access '/tmp/backups/*': No such file or directory
[tiredofit/db-backup:4.0.27 09:32:48 /] $ ls -l /tmp/backups/
total 0
[tiredofit/db-backup:4.0.27 09:32:51 /] $ sudo -Eu dbbackup pigz -q -3 -p 8 /tmp/backups/mssql_test1_example-db-host_20231204-085444.bak
[tiredofit/db-backup:4.0.27 09:33:13 /] $ ls -l /tmp/backups/*
ls: cannot access '/tmp/backups/*': No such file or directory
[tiredofit/db-backup:4.0.27 09:34:20 /] $ ls -l /backup/*
-rw------- 1 root  root      0 Dec  4 08:51 /backup/mssql_test1_example-db-host_20231204-085144.bak.gz.sha1
-rw------- 1 root  root      0 Dec  4 08:53 /backup/mssql_test1_example-db-host_20231204-085324.bak.gz.sha1
-rw------- 1 10001 root 369348 Dec  4 08:54 /backup/mssql_test1_example-db-host_20231204-085444.bak.gz
-rw------- 1 root  root     93 Dec  4 08:54 /backup/mssql_test1_example-db-host_20231204-085444.bak.gz.sha1
-rw------- 1 10001 root 369387 Dec  4 09:12 /backup/mssql_test1_example-db-host_20231204-091250.bak.gz
-rw------- 1 root  root     93 Dec  4 09:12 /backup/mssql_test1_example-db-host_20231204-091250.bak.gz.sha1
-rw------- 1 10001 root 369453 Dec  4 09:15 /backup/mssql_test1_example-db-host_20231204-091459.bak.gz
-rw------- 1 root  root     93 Dec  4 09:15 /backup/mssql_test1_example-db-host_20231204-091459.bak.gz.sha1
tiredofit commented 10 months ago

OK, interesting, Compression is actually working for you here, so I don't see a compression issue.

Perhaps I can get a DEBUG_MOVE_DBBACKUP=TRUE which will give us some more insight into the blobxfer command.

lazarevdmitriy commented 10 months ago 
docker-compose up

[+] Building 0.0s (0/0)                                                                                                                                  docker:desktop-linux
[+] Running 2/2
 ✔ Container example-mssql-s3-db               Created                                                                                                                   0.0s
 ✔ Container example-mssql-blobxfer-db-backup  Recreated                                                                                                                 0.1s
Attaching to example-mssql-blobxfer-db-backup, example-mssql-s3-db
example-mssql-s3-db               | SQL Server 2019 will run as non-root by default.
example-mssql-s3-db               | This container is running as user mssql.
example-mssql-s3-db               | Your master database file is owned by mssql.
example-mssql-s3-db               | To learn more visit https://go.microsoft.com/fwlink/?linkid=2099216.
example-mssql-blobxfer-db-backup  | s6-linux-init: warning: parent wrote to fd 3!
example-mssql-blobxfer-db-backup  |                                                                        ,---.
example-mssql-blobxfer-db-backup  | ,--------.,--.                 ,--.            ,---.    ,--. ,--------.|   |
example-mssql-blobxfer-db-backup  | '--.  .--'`--',--.--. ,---.  ,-|  |     ,---. /  .-'    |  | '--.  .--'|  .'
example-mssql-blobxfer-db-backup  |    |  |   ,--.|  .--'| .-. :' .-. |    | .-. ||  `-,    |  |    |  |   |  |
example-mssql-blobxfer-db-backup  |    |  |   |  ||  |   \   --.\ `-' |    ' '-' '|  .-'    |  |.--.|  |   `--'
example-mssql-blobxfer-db-backup  |    `--'   `--'`--'    `----' `---'      `---' `--'      `--''--'`--'   .--.
example-mssql-blobxfer-db-backup  |                                                                        '--'
example-mssql-blobxfer-db-backup  | Image:  tiredofit/db-backup | Version  4.0.27 Type 'image_changelog' for details
example-mssql-blobxfer-db-backup  | Repository/Documentation: https://github.com/tiredofit/docker-db-backup/
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | If this image provides you value  - Consider sponsoring my work for continued
example-mssql-blobxfer-db-backup  | development, timely updates, and feature requests. Commercial support available.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  |                     More Info:  https://www.tiredofit.ca
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | 2023-12-04.17:41:48 [NOTICE] ** [timezone] Timezone: Setting to 'America/Vancouver' from 'Etc/GMT'
example-mssql-blobxfer-db-backup  | 2023-12-04.09:41:50 [NOTICE] ** [scheduling] Container configured for scheduled tasks with 'cron'
example-mssql-blobxfer-db-backup  | 2023-12-04.09:41:52 [NOTICE] ** [messaging] Container configured to route mail via SMTP to 'postfix-relay'
example-mssql-s3-db               | 2023-12-04 17:41:55.31 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.mdf' to '/var/opt/mssql/data/model_replicatedmaster.mdf'.
2023-12-04 17:41:55.36 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.ldf' to '/var/opt/mssql/data/model_replicatedmaster.ldf'.
2023-12-04 17:41:55.37 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdbdata.mdf' to '/var/opt/mssql/data/model_msdbdata.mdf'.
2023-12-04 17:41:55.43 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdblog.ldf' to '/var/opt/mssql/data/model_msdblog.ldf'.
2023-12-04 17:41:55.49 Server      Microsoft SQL Server 2019 (RTM-CU23) (KB5030333) - 15.0.4335.1 (X64)
example-Sep 21 2023 17:28:44      |
example-Copyright (C) 2019 Microsoft Corporation
example-Express Edition (64-bit) on Linux (Ubuntu 20.04.6 LTS) <X64>
2023-12-04 17:41:55.49 Server      UTC adjustment: 0:00
2023-12-04 17:41:55.49 Server      (c) Microsoft Corporation.
2023-12-04 17:41:55.49 Server      All rights reserved.
2023-12-04 17:41:55.49 Server      Server process ID is 388.
2023-12-04 17:41:55.49 Server      Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2023-12-04 17:41:55.49 Server      Registry startup parameters:
example- -d /var/opt/mssql/data/master.mdf
example- -l /var/opt/mssql/data/mastlog.ldf
example- -e /var/opt/mssql/log/errorlog
2023-12-04 17:41:55.50 Server      SQL Server detected 1 sockets with 8 cores per socket and 8 logical processors per socket, 8 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required.
2023-12-04 17:41:55.50 Server      SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.
2023-12-04 17:41:55.50 Server      Detected 3141 MB of RAM. This is an informational message; no user action is required.
2023-12-04 17:41:55.51 Server      Using conventional memory in the memory manager.
2023-12-04 17:41:55.51 Server      Page exclusion bitmap is enabled.
2023-12-04 17:41:55.55 Server      Buffer pool extension is not supported on Linux platform.
2023-12-04 17:41:55.55 Server      Buffer Pool: Allocating 524288 bytes for 396316 hashPages.
2023-12-04 17:41:55.68 Server      Buffer pool extension is already disabled. No action is necessary.
example-mssql-blobxfer-db-backup  | 2023-12-04.09:41:55 [STARTING] ** [scheduling] [1] Starting cron
2023-12-04 17:41:55.92 Server      Successfully initialized the TLS configuration. Allowed TLS protocol versions are ['1.0 1.1 1.2']. Allowed TLS ciphers are ['ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!DHE-RSA-AES256-GCM-SHA384:!DHE-RSA-AES128-GCM-SHA256:!DHE-RSA-AES256-SHA:!DHE-RSA-AES128-SHA'].
2023-12-04 17:41:55.97 Server      Query Store settings initialized with enabled = 1,
SetThreadAffinity failed: 87      |
2023-12-04 17:41:56.00 Server      The maximum number of dedicated administrator connections for this instance is '1'
2023-12-04 17:41:56.00 Server      Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2023-12-04 17:41:56.01 Server      Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This is an informational message only.  No user action is required.
2023-12-04 17:41:56.06 Server      In-Memory OLTP initialized on lowend machine.
2023-12-04 17:41:56.15 Server      [INFO] Created Extended Events session 'hkenginexesession'
2023-12-04 17:41:56.16 Server      Database Instant File Initialization: enabled. For security and performance considerations see the topic 'Database Instant File Initialization' in SQL Server Books Online. This is an informational message only. No user action is required.
ForceFlush is enabled for this instance.
2023-12-04 17:41:56.17 Server      Total Log Writer threads: 2. This is an informational message; no user action is required.
2023-12-04 17:41:56.19 Server      Software Usage Metrics is disabled.
2023-12-04 17:41:56.23 spid9s      [1]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 17:41:56.23 spid9s      Starting up database 'master'.
ForceFlush feature is enabled for log durability.
2023-12-04 17:41:56.30 Server      CLR version v4.0.30319 loaded.
2023-12-04 17:41:56.60 spid9s      SQL Server Audit is starting the audits. This is an informational message. No user action is required.
2023-12-04 17:41:56.60 spid9s      SQL Server Audit has started the audits. This is an informational message. No user action is required.
2023-12-04 17:41:56.69 spid9s      SQL Trace ID 1 was started by login "sa".
2023-12-04 17:41:56.76 spid9s      Server name is 'example-db-host'. This is an informational message only. No user action is required.
2023-12-04 17:41:56.80 spid23s     Always On: The availability replica manager is starting. This is an informational message only. No user action is required.
2023-12-04 17:41:56.81 spid9s      [4]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 17:41:56.81 spid23s     Always On: The availability replica manager is waiting for the instance of SQL Server to allow client connections. This is an informational message only. No user action is required.
2023-12-04 17:41:56.81 spid9s      Starting up database 'msdb'.
2023-12-04 17:41:56.82 spid12s     [32767]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 17:41:56.82 spid12s     Starting up database 'mssqlsystemresource'.
2023-12-04 17:41:56.83 spid12s     The resource database build version is 15.00.4335. This is an informational message only. No user action is required.
2023-12-04 17:41:56.89 spid12s     [3]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 17:41:56.89 spid12s     Starting up database 'model'.
2023-12-04 17:41:56.94 spid22s     A self-generated certificate was successfully loaded for encryption.
2023-12-04 17:41:56.97 spid22s     Server is listening on [ 'any' <ipv6> 1433].
2023-12-04 17:41:56.97 spid22s     Server is listening on [ 'any' <ipv4> 1433].
2023-12-04 17:41:56.98 spid22s     Dedicated administrator connection support was not started because it is disabled on this edition of SQL Server. If you want to use a dedicated administrator connection, restart SQL Server using the trace flag 7806. This is an informational message only. No user action is required.
2023-12-04 17:41:56.98 spid12s     Clearing tempdb database.
2023-12-04 17:41:57.00 spid22s     Server is listening on [ ::1 <ipv6> 1431].
2023-12-04 17:41:57.00 spid22s     Server is listening on [ 127.0.0.1 <ipv4> 1431].
2023-12-04 17:41:57.01 spid22s     SQL Server is now ready for client connections. This is an informational message; no user action is required.
2023-12-04 17:41:57.09 spid12s     [2]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 17:41:57.09 spid12s     Starting up database 'tempdb'.
2023-12-04 17:41:57.24 spid23s     The Service Broker endpoint is in disabled or stopped state.
2023-12-04 17:41:57.24 spid23s     The Database Mirroring endpoint is in disabled or stopped state.
2023-12-04 17:41:57.33 spid23s     Service Broker manager has started.
2023-12-04 17:41:57.34 spid9s      Recovery is complete. This is an informational message only. No user action is required.
2023-12-04 17:41:58.01 Server      Common language runtime (CLR) functionality initialized.
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:00 [INFO] ** [01-example-db-host__test1] Next Backup at 2023-12-04 09:42:00 PST
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:00 [NOTICE] ** [01-example-db-host__test1] Dumping MSSQL database: ''
2023-12-04 17:42:00.87 spid51      [5]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 17:42:00.87 spid51      Starting up database 'test1'.
2023-12-04 17:42:00.91 spid51      Parallel redo is started for database 'test1' with worker pool size [2].
2023-12-04 17:42:00.93 spid51      Parallel redo is shutdown for database 'test1' with worker pool size [2].
2023-12-04 17:42:01.01 Backup      Database backed up. Database: test1, creation date(time): 2023/12/04(16:54:06), pages dumped: 339, first LSN: 36:944:1, last LSN: 36:968:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {'/tmp/backups/mssql_test1_example-db-host_20231204-094200.bak'}). This is an informational message only. No user action is required.
2023-12-04 17:42:01.10 Backup      BACKUP DATABASE successfully processed 330 pages in 0.032 seconds (80.444 MB/sec).
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:01 [INFO] ** [01-example-db-host__test1] DB Backup of 'mssql_test1_example-db-host_20231204-094200.bak.gz' completed successfully
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:01 [NOTICE] ** [01-example-db-host__test1] Generating SHA1 sum for 'mssql_test1_example-db-host_20231204-094200.bak.gz'
example-mssql-blobxfer-db-backup  | + '[' 0 = 0 ']'
example-mssql-blobxfer-db-backup  | ++ run_as_user stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | ++ sudo -Eu dbbackup stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + dbbackup_size=369520
example-mssql-blobxfer-db-backup  | ++ run_as_user date -r /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz +%s
example-mssql-blobxfer-db-backup  | ++ sudo -Eu dbbackup date -r /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz +%s
example-mssql-blobxfer-db-backup  | + dbbackup_date=1701711721
example-mssql-blobxfer-db-backup  | + case "${backup_job_size_value,,}" in
example-mssql-blobxfer-db-backup  | + backup_job_size_value=1
example-mssql-blobxfer-db-backup  | + '[' 1 = 1 ']'
example-mssql-blobxfer-db-backup  | ++ run_as_user stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | ++ sudo -Eu dbbackup stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + filesize=369520
example-mssql-blobxfer-db-backup  | + write_log notice 'Backup of '\''mssql_test1_example-db-host_20231204-094200.bak.gz'\'' created with the size of 369520 bytes'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=notice
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=Backup of '\''mssql_test1_example-db-host_20231204-094200.bak.gz'\'' created with the size of 369520 bytes'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 09:42:01 PST [notice] Backup of '\''mssql_test1_example-db-host_20231204-094200.bak.gz'\'' created with the size of 369520 bytes'
example-mssql-blobxfer-db-backup  | ++ date -d @1701711720 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_094200-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_094200-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_notice 'Backup of '\''mssql_test1_example-db-host_20231204-094200.bak.gz'\'' created with the size of 369520 bytes'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + case "${CONTAINER_LOG_LEVEL,,}" in
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.09:42:01 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.09:42:01 \e[100m[NOTICE]\e[49m ** [01-example-db-host__test1] Backup of '\''mssql_test1_example-db-host_20231204-094200.bak.gz'\'' created with the size of 369520 bytes'
example-mssql-blobxfer-db-backup  | + case "${CONTAINER_LOG_FILE_LEVEL,,}" in
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:01 [NOTICE] ** [01-example-db-host__test1] Backup of 'mssql_test1_example-db-host_20231204-094200.bak.gz' created with the size of 369520 bytes
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.09:42:01 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.09:42:01 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] Backup of '\''mssql_test1_example-db-host_20231204-094200.bak.gz'\'' created with the size of 369520 bytes'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + chmod 600 /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + case "${backup_job_backup_location,,}" in
example-mssql-blobxfer-db-backup  | + write_log info 'Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=info
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 09:42:01 PST [info] Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | ++ date -d @1701711720 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_094200-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_094200-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_info 'Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.09:42:01 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.09:42:01 \e[42m[INFO]\e[49m ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:01 [INFO] ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.09:42:02 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.09:42:02 [INFO] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + blobxfer download --mode file --remote-path docker-db-backup --storage-account ''\''"mikhailmharbortest"'\''' --storage-account-key --local-path /backup --delete
example-mssql-blobxfer-db-backup  | Usage: blobxfer download [OPTIONS]
example-mssql-blobxfer-db-backup  | Try 'blobxfer download -h' for help.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | Error: Got unexpected extra argument (/backup)
example-mssql-blobxfer-db-backup  | + write_log info 'Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=info
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 09:42:02 PST [info] Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | ++ date -d @1701711720 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_094200-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_094200-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_info 'Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.09:42:02 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:02 [INFO] ** [01-example-db-host__test1] Moving backup to external storage with blobxfer
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.09:42:02 \e[42m[INFO]\e[49m ** [01-example-db-host__test1] Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.09:42:02 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.09:42:02 [INFO] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + mkdir -p /backup
example-mssql-blobxfer-db-backup  | + '[' SHA1 '!=' none ']'
example-mssql-blobxfer-db-backup  | + run_as_user mv /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz.sha1 /backup/
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup mv /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz.sha1 /backup/
example-mssql-blobxfer-db-backup  | + run_as_user mv /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz /backup/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup mv /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz /backup/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + silent run_as_user blobxfer upload --mode file --remote-path docker-db-backup --storage-account ''\''"mikhailmharbortest"'\''' --storage-account-key --local-path /backup
example-mssql-blobxfer-db-backup  | + '[' FALSE = true ']'
example-mssql-blobxfer-db-backup  | + '[' true = true ']'
example-mssql-blobxfer-db-backup  | + run_as_user blobxfer upload --mode file --remote-path docker-db-backup --storage-account ''\''"mikhailmharbortest"'\''' --storage-account-key --local-path /backup
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup blobxfer upload --mode file --remote-path docker-db-backup --storage-account ''\''"mikhailmharbortest"'\''' --storage-account-key --local-path /backup
example-mssql-blobxfer-db-backup  | Usage: blobxfer upload [OPTIONS]
example-mssql-blobxfer-db-backup  | Try 'blobxfer upload -h' for help.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | Error: Got unexpected extra argument (/backup)
example-mssql-blobxfer-db-backup  | + move_exit_code=2
example-mssql-blobxfer-db-backup  | + '[' SHA1 '!=' none ']'
example-mssql-blobxfer-db-backup  | + run_as_user rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz.sha1
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz.sha1
example-mssql-blobxfer-db-backup  | + run_as_user rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + run_as_user rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-094200.bak.gz
example-mssql-blobxfer-db-backup  | + var_true TRUE
example-mssql-blobxfer-db-backup  | + '[' true = true ']'
example-mssql-blobxfer-db-backup  | + debug off
example-mssql-blobxfer-db-backup  | + case "${1}" in
example-mssql-blobxfer-db-backup  | + backup_job_log_level='notice}'
example-mssql-blobxfer-db-backup  | + CONTAINER_LOG_LEVEL=NOTICE
example-mssql-blobxfer-db-backup  | + DEBUG_MODE=FALSE
example-mssql-blobxfer-db-backup  | + SHOW_OUTPUT=
example-mssql-blobxfer-db-backup  | + var_true FALSE
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = yes ']'
example-mssql-blobxfer-db-backup  | + set +x
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:03 [ERROR] ** [01-example-db-host__test1] Moving of backup 'mssql_test1_example-db-host_20231204-094200.bak.gz' reported errors
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:03 [NOTICE] ** [01-example-db-host__test1] DB Backup for 'test1' time taken: Hours: 0 Minutes: 00 Seconds: 03
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:03 [ERROR] ** [01-example-db-host__test1] Skipping Cleaning up old backups because there were errors in backing up
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:03 [INFO] ** [01-example-db-host__test1] Backup 01 routines finish time: 2023-12-04 09:42:03 PST with exit code 0
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:03 [NOTICE] ** [01-example-db-host__test1] Backup 01 routines time taken: Hours: 0 Minutes: 00 Seconds: 03
example-mssql-blobxfer-db-backup  | 2023-12-04.09:42:03 [NOTICE] ** [01-example-db-host__test1] Sleeping for another 86397 seconds. Waking up at 2023-12-05 09:42:00 PST
tiredofit commented 10 months ago

OK, I see your problem. Check your compose file and change SB01_BLOBXFER_STORAGE_ACCOUNT_KEY to DB01_BLOBXFER_STORAGE_ACCOUNT_KEY

lazarevdmitriy commented 10 months ago

Please fix this line https://github.com/tiredofit/docker-db-backup/blob/2cf3e2ae70fdfa15b13eb6450c2a55e9163277bf/examples/mssql-blobxfer/compose.yml#L60

tiredofit commented 10 months ago

Aha! Straight out of the examples. My apologies..

lazarevdmitriy commented 10 months ago

I have another error

docker-compose up

[+] Building 0.0s (0/0)                                                                                                                                  docker:desktop-linux
[+] Running 2/2
 ✔ Container example-mssql-s3-db               Created                                                                                                                   0.0s
 ✔ Container example-mssql-blobxfer-db-backup  Recreated                                                                                                                 0.1s
Attaching to example-mssql-blobxfer-db-backup, example-mssql-s3-db
example-mssql-s3-db               | SQL Server 2019 will run as non-root by default.
example-mssql-s3-db               | This container is running as user mssql.
example-mssql-s3-db               | Your master database file is owned by mssql.
example-mssql-s3-db               | To learn more visit https://go.microsoft.com/fwlink/?linkid=2099216.
example-mssql-blobxfer-db-backup  | s6-linux-init: warning: parent wrote to fd 3!
example-mssql-blobxfer-db-backup  |                                                                        ,---.
example-mssql-blobxfer-db-backup  | ,--------.,--.                 ,--.            ,---.    ,--. ,--------.|   |
example-mssql-blobxfer-db-backup  | '--.  .--'`--',--.--. ,---.  ,-|  |     ,---. /  .-'    |  | '--.  .--'|  .'
example-mssql-blobxfer-db-backup  |    |  |   ,--.|  .--'| .-. :' .-. |    | .-. ||  `-,    |  |    |  |   |  |
example-mssql-blobxfer-db-backup  |    |  |   |  ||  |   \   --.\ `-' |    ' '-' '|  .-'    |  |.--.|  |   `--'
example-mssql-blobxfer-db-backup  |    `--'   `--'`--'    `----' `---'      `---' `--'      `--''--'`--'   .--.
example-mssql-blobxfer-db-backup  |                                                                        '--'
example-mssql-blobxfer-db-backup  | Image:  tiredofit/db-backup | Version  4.0.27 Type 'image_changelog' for details
example-mssql-blobxfer-db-backup  | Repository/Documentation: https://github.com/tiredofit/docker-db-backup/
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | If this image provides you value  - Consider sponsoring my work for continued
example-mssql-blobxfer-db-backup  | development, timely updates, and feature requests. Commercial support available.
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  |                     More Info:  https://www.tiredofit.ca
example-mssql-blobxfer-db-backup  |
example-mssql-blobxfer-db-backup  | 2023-12-04.18:01:31 [NOTICE] ** [timezone] Timezone: Setting to 'America/Vancouver' from 'Etc/GMT'
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:33 [NOTICE] ** [scheduling] Container configured for scheduled tasks with 'cron'
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:34 [NOTICE] ** [messaging] Container configured to route mail via SMTP to 'postfix-relay'
example-mssql-s3-db               | 2023-12-04 18:01:35.97 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.mdf' to '/var/opt/mssql/data/model_replicatedmaster.mdf'.
2023-12-04 18:01:36.03 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_replicatedmaster.ldf' to '/var/opt/mssql/data/model_replicatedmaster.ldf'.
2023-12-04 18:01:36.05 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdbdata.mdf' to '/var/opt/mssql/data/model_msdbdata.mdf'.
2023-12-04 18:01:36.11 Server      Setup step is FORCE copying system data file 'C:\templatedata\model_msdblog.ldf' to '/var/opt/mssql/data/model_msdblog.ldf'.
2023-12-04 18:01:36.16 Server      Microsoft SQL Server 2019 (RTM-CU23) (KB5030333) - 15.0.4335.1 (X64)
example-Sep 21 2023 17:28:44      |
example-Copyright (C) 2019 Microsoft Corporation
example-Express Edition (64-bit) on Linux (Ubuntu 20.04.6 LTS) <X64>
2023-12-04 18:01:36.17 Server      UTC adjustment: 0:00
2023-12-04 18:01:36.17 Server      (c) Microsoft Corporation.
2023-12-04 18:01:36.17 Server      All rights reserved.
2023-12-04 18:01:36.17 Server      Server process ID is 384.
2023-12-04 18:01:36.17 Server      Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2023-12-04 18:01:36.17 Server      Registry startup parameters:
example- -d /var/opt/mssql/data/master.mdf
example- -l /var/opt/mssql/data/mastlog.ldf
example- -e /var/opt/mssql/log/errorlog
2023-12-04 18:01:36.17 Server      SQL Server detected 1 sockets with 8 cores per socket and 8 logical processors per socket, 8 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required.
2023-12-04 18:01:36.18 Server      SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.
2023-12-04 18:01:36.18 Server      Detected 3141 MB of RAM. This is an informational message; no user action is required.
2023-12-04 18:01:36.18 Server      Using conventional memory in the memory manager.
2023-12-04 18:01:36.19 Server      Page exclusion bitmap is enabled.
2023-12-04 18:01:36.22 Server      Buffer pool extension is not supported on Linux platform.
2023-12-04 18:01:36.22 Server      Buffer Pool: Allocating 524288 bytes for 395740 hashPages.
2023-12-04 18:01:36.34 Server      Buffer pool extension is already disabled. No action is necessary.
2023-12-04 18:01:36.58 Server      Successfully initialized the TLS configuration. Allowed TLS protocol versions are ['1.0 1.1 1.2']. Allowed TLS ciphers are ['ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!DHE-RSA-AES256-GCM-SHA384:!DHE-RSA-AES128-GCM-SHA256:!DHE-RSA-AES256-SHA:!DHE-RSA-AES128-SHA'].
2023-12-04 18:01:36.65 Server      Query Store settings initialized with enabled = 1,
SetThreadAffinity failed: 87      |
2023-12-04 18:01:36.69 Server      The maximum number of dedicated administrator connections for this instance is '1'
2023-12-04 18:01:36.69 Server      Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2023-12-04 18:01:36.71 Server      Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This is an informational message only.  No user action is required.
2023-12-04 18:01:36.74 Server      In-Memory OLTP initialized on lowend machine.
2023-12-04 18:01:36.96 Server      [INFO] Created Extended Events session 'hkenginexesession'
2023-12-04 18:01:36.97 Server      Database Instant File Initialization: enabled. For security and performance considerations see the topic 'Database Instant File Initialization' in SQL Server Books Online. This is an informational message only. No user action is required.
ForceFlush is enabled for this instance.
2023-12-04 18:01:36.99 Server      Total Log Writer threads: 2. This is an informational message; no user action is required.
2023-12-04 18:01:37.01 Server      Software Usage Metrics is disabled.
2023-12-04 18:01:37.04 spid11s     [1]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 18:01:37.05 spid11s     Starting up database 'master'.
ForceFlush feature is enabled for log durability.
2023-12-04 18:01:37.11 Server      CLR version v4.0.30319 loaded.
2023-12-04 18:01:37.57 spid11s     SQL Server Audit is starting the audits. This is an informational message. No user action is required.
2023-12-04 18:01:37.59 spid11s     SQL Server Audit has started the audits. This is an informational message. No user action is required.
2023-12-04 18:01:37.74 spid11s     SQL Trace ID 1 was started by login "sa".
2023-12-04 18:01:37.83 spid11s     Server name is 'example-db-host'. This is an informational message only. No user action is required.
2023-12-04 18:01:37.89 spid23s     Always On: The availability replica manager is starting. This is an informational message only. No user action is required.
2023-12-04 18:01:37.90 spid23s     Always On: The availability replica manager is waiting for the instance of SQL Server to allow client connections. This is an informational message only. No user action is required.
2023-12-04 18:01:37.91 spid11s     [4]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 18:01:37.91 spid11s     Starting up database 'msdb'.
2023-12-04 18:01:37.92 spid12s     [32767]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 18:01:37.93 spid12s     Starting up database 'mssqlsystemresource'.
2023-12-04 18:01:38.00 spid12s     The resource database build version is 15.00.4335. This is an informational message only. No user action is required.
2023-12-04 18:01:38.12 spid12s     [3]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 18:01:38.13 spid12s     Starting up database 'model'.
2023-12-04 18:01:38.28 spid22s     A self-generated certificate was successfully loaded for encryption.
2023-12-04 18:01:38.29 spid12s     Clearing tempdb database.
2023-12-04 18:01:38.33 spid22s     Server is listening on [ 'any' <ipv6> 1433].
2023-12-04 18:01:38.34 spid22s     Server is listening on [ 'any' <ipv4> 1433].
2023-12-04 18:01:38.36 spid22s     Dedicated administrator connection support was not started because it is disabled on this edition of SQL Server. If you want to use a dedicated administrator connection, restart SQL Server using the trace flag 7806. This is an informational message only. No user action is required.
2023-12-04 18:01:38.38 spid22s     Server is listening on [ ::1 <ipv6> 1431].
2023-12-04 18:01:38.39 spid22s     Server is listening on [ 127.0.0.1 <ipv4> 1431].
2023-12-04 18:01:38.40 spid22s     SQL Server is now ready for client connections. This is an informational message; no user action is required.
2023-12-04 18:01:38.48 spid12s     [2]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 18:01:38.48 spid12s     Starting up database 'tempdb'.
2023-12-04 18:01:38.58 spid23s     The Service Broker endpoint is in disabled or stopped state.
2023-12-04 18:01:38.58 spid23s     The Database Mirroring endpoint is in disabled or stopped state.
2023-12-04 18:01:38.73 spid23s     Service Broker manager has started.
2023-12-04 18:01:38.74 spid11s     Recovery is complete. This is an informational message only. No user action is required.
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:38 [STARTING] ** [scheduling] [1] Starting cron
2023-12-04 18:01:39.58 Server      Common language runtime (CLR) functionality initialized.
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:43 [INFO] ** [01-example-db-host__test1] Next Backup at 2023-12-04 10:01:42 PST
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:43 [NOTICE] ** [01-example-db-host__test1] Dumping MSSQL database: ''
2023-12-04 18:01:43.56 spid51      [5]. Feature Status: PVS: 0. CTR: 0. ConcurrentPFSUpdate: 1.
2023-12-04 18:01:43.56 spid51      Starting up database 'test1'.
2023-12-04 18:01:43.60 spid51      Parallel redo is started for database 'test1' with worker pool size [2].
2023-12-04 18:01:43.62 spid51      Parallel redo is shutdown for database 'test1' with worker pool size [2].
2023-12-04 18:01:43.72 Backup      Database backed up. Database: test1, creation date(time): 2023/12/04(16:54:06), pages dumped: 339, first LSN: 36:1520:1, last LSN: 36:1544:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {'/tmp/backups/mssql_test1_example-db-host_20231204-100143.bak'}). This is an informational message only. No user action is required.
2023-12-04 18:01:43.82 Backup      BACKUP DATABASE successfully processed 330 pages in 0.033 seconds (78.006 MB/sec).
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:44 [INFO] ** [01-example-db-host__test1] DB Backup of 'mssql_test1_example-db-host_20231204-100143.bak.gz' completed successfully
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:44 [NOTICE] ** [01-example-db-host__test1] Generating SHA1 sum for 'mssql_test1_example-db-host_20231204-100143.bak.gz'
example-mssql-blobxfer-db-backup  | + '[' 0 = 0 ']'
example-mssql-blobxfer-db-backup  | ++ run_as_user stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | ++ sudo -Eu dbbackup stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + dbbackup_size=369727
example-mssql-blobxfer-db-backup  | ++ run_as_user date -r /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz +%s
example-mssql-blobxfer-db-backup  | ++ sudo -Eu dbbackup date -r /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz +%s
example-mssql-blobxfer-db-backup  | + dbbackup_date=1701712903
example-mssql-blobxfer-db-backup  | + case "${backup_job_size_value,,}" in
example-mssql-blobxfer-db-backup  | + backup_job_size_value=1
example-mssql-blobxfer-db-backup  | + '[' 1 = 1 ']'
example-mssql-blobxfer-db-backup  | ++ run_as_user stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | ++ sudo -Eu dbbackup stat -c%s /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + filesize=369727
example-mssql-blobxfer-db-backup  | + write_log notice 'Backup of '\''mssql_test1_example-db-host_20231204-100143.bak.gz'\'' created with the size of 369727 bytes'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=notice
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=Backup of '\''mssql_test1_example-db-host_20231204-100143.bak.gz'\'' created with the size of 369727 bytes'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 10:01:44 PST [notice] Backup of '\''mssql_test1_example-db-host_20231204-100143.bak.gz'\'' created with the size of 369727 bytes'
example-mssql-blobxfer-db-backup  | ++ date -d @1701712903 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_100143-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_100143-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_notice 'Backup of '\''mssql_test1_example-db-host_20231204-100143.bak.gz'\'' created with the size of 369727 bytes'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + case "${CONTAINER_LOG_LEVEL,,}" in
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.10:01:44 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:44 [NOTICE] ** [01-example-db-host__test1] Backup of 'mssql_test1_example-db-host_20231204-100143.bak.gz' created with the size of 369727 bytes
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.10:01:44 \e[100m[NOTICE]\e[49m ** [01-example-db-host__test1] Backup of '\''mssql_test1_example-db-host_20231204-100143.bak.gz'\'' created with the size of 369727 bytes'
example-mssql-blobxfer-db-backup  | + case "${CONTAINER_LOG_FILE_LEVEL,,}" in
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.10:01:44 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.10:01:44 [NOTICE] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] Backup of '\''mssql_test1_example-db-host_20231204-100143.bak.gz'\'' created with the size of 369727 bytes'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + chmod 600 /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + case "${backup_job_backup_location,,}" in
example-mssql-blobxfer-db-backup  | + write_log info 'Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=info
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 10:01:44 PST [info] Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | ++ date -d @1701712903 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_100143-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_100143-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_info 'Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.10:01:44 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:44 [INFO] ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.10:01:44 \e[42m[INFO]\e[49m ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.10:01:44 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.10:01:44 [INFO] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] Synchronize local storage from S3 Bucket with blobxfer'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + blobxfer download --mode file --remote-path docker-db-backup --storage-account ''\''"blobxfertest"'\''' --storage-account-key ''\''"fjJswxR7u4564356457645754S/sjBsQDrBZVFuC+rDPlPW88zPbzGNHkuFn46ETlh5aPZZ3B2342352345dEEg=="'\''' --local-path /backup --delete
example-mssql-blobxfer-db-backup  | Traceback (most recent call last):
example-mssql-blobxfer-db-backup  |   File "/usr/bin/blobxfer", line 8, in <module>
example-mssql-blobxfer-db-backup  |     sys.exit(cli())
example-mssql-blobxfer-db-backup  |              ^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
example-mssql-blobxfer-db-backup  |     return self.main(*args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1078, in main
example-mssql-blobxfer-db-backup  |     rv = self.invoke(ctx)
example-mssql-blobxfer-db-backup  |          ^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
example-mssql-blobxfer-db-backup  |     return _process_result(sub_ctx.command.invoke(sub_ctx))
example-mssql-blobxfer-db-backup  |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
example-mssql-blobxfer-db-backup  |     return ctx.invoke(self.callback, **ctx.params)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 783, in invoke
example-mssql-blobxfer-db-backup  |     return __callback(*args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/decorators.py", line 92, in new_func
example-mssql-blobxfer-db-backup  |     return ctx.invoke(f, obj, *args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 783, in invoke
example-mssql-blobxfer-db-backup  |     return __callback(*args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer_cli/cli.py", line 1094, in download
example-mssql-blobxfer-db-backup  |     ctx.initialize(settings.TransferAction.Download)
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer_cli/cli.py", line 71, in initialize
example-mssql-blobxfer-db-backup  |     self.credentials = settings.create_azure_storage_credentials(
example-mssql-blobxfer-db-backup  |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer_cli/settings.py", line 425, in create_azure_storage_credentials
example-mssql-blobxfer-db-backup  |     creds.add_storage_account(name, key, endpoint)
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer/operations/azure/__init__.py", line 69, in add_storage_account
example-mssql-blobxfer-db-backup  |     self._storage_accounts[name] = StorageAccount(
example-mssql-blobxfer-db-backup  |                                    ^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer/operations/azure/__init__.py", line 136, in __init__
example-mssql-blobxfer-db-backup  |     raise ValueError(
example-mssql-blobxfer-db-backup  | ValueError: specified storage account key is invalid for storage account: '"blobxfertest"'
example-mssql-blobxfer-db-backup  | + write_log info 'Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + local _arg_log_level=info
example-mssql-blobxfer-db-backup  | + shift 1
example-mssql-blobxfer-db-backup  | + local '_arg_log_message=Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + '[' -n mssql ']'
example-mssql-blobxfer-db-backup  | + '[' -n test1 ']'
example-mssql-blobxfer-db-backup  | + '[' -n example-db-host ']'
example-mssql-blobxfer-db-backup  | + '[' -n mssql_test1_example-db-host ']'
example-mssql-blobxfer-db-backup  | + case "${_arg_log_level,,}" in
example-mssql-blobxfer-db-backup  | ++ date '+%Y-%m-%d %H:%M:%S %Z'
example-mssql-blobxfer-db-backup  | ++ date +%Y%m%d
example-mssql-blobxfer-db-backup  | + echo '2023-12-04 10:01:45 PST [info] Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | ++ date -d @1701712903 +%Y%m%d_%H%M%S
example-mssql-blobxfer-db-backup  | + run_as_user tee -a /logs/20231204/20231204_100143-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup tee -a /logs/20231204/20231204_100143-mssql_test1_example-db-host.log
example-mssql-blobxfer-db-backup  | + print_info 'Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + output_off
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = false ']'
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.10:01:45 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.10:01:45 \e[42m[INFO]\e[49m ** [01-example-db-host__test1] Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:45 [INFO] ** [01-example-db-host__test1] Moving backup to external storage with blobxfer
example-mssql-blobxfer-db-backup  | ++ log_prefix
example-mssql-blobxfer-db-backup  | ++ output_off
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ '[' true = true ']'
example-mssql-blobxfer-db-backup  | +++ date +%Y-%m-%d
example-mssql-blobxfer-db-backup  | +++ date +%H:%M:%S
example-mssql-blobxfer-db-backup  | ++ echo '2023-12-04.10:01:45 '
example-mssql-blobxfer-db-backup  | ++ output_on
example-mssql-blobxfer-db-backup  | ++ '[' false = true ']'
example-mssql-blobxfer-db-backup  | ++ basename ./run
example-mssql-blobxfer-db-backup  | + echo -e '2023-12-04.10:01:45 [INFO] /etc/services.available/dbbackup-01/run ** [01-example-db-host__test1] Moving backup to external storage with blobxfer'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + output_on
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + var_true ''
example-mssql-blobxfer-db-backup  | + '[' '' = true ']'
example-mssql-blobxfer-db-backup  | + '[' '' = yes ']'
example-mssql-blobxfer-db-backup  | + mkdir -p /backup
example-mssql-blobxfer-db-backup  | + '[' SHA1 '!=' none ']'
example-mssql-blobxfer-db-backup  | + run_as_user mv /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz.sha1 /backup/
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup mv /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz.sha1 /backup/
example-mssql-blobxfer-db-backup  | + run_as_user mv /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz /backup/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup mv /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz /backup/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + silent run_as_user blobxfer upload --mode file --remote-path docker-db-backup --storage-account ''\''"blobxfertest"'\''' --storage-account-key ''\''"fjJswxR7u4564356457645754S/sjBsQDrBZVFuC+rDPlPW88zPbzGNHkuFn46ETlh5aPZZ3B2342352345dEEg=="'\''' --local-path /backup
example-mssql-blobxfer-db-backup  | + '[' FALSE = true ']'
example-mssql-blobxfer-db-backup  | + '[' true = true ']'
example-mssql-blobxfer-db-backup  | + run_as_user blobxfer upload --mode file --remote-path docker-db-backup --storage-account ''\''"blobxfertest"'\''' --storage-account-key ''\''"fjJswxR7u4564356457645754S/sjBsQDrBZVFuC+rDPlPW88zPbzGNHkuFn46ETlh5aPZZ3B2342352345dEEg=="'\''' --local-path /backup
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup blobxfer upload --mode file --remote-path docker-db-backup --storage-account ''\''"blobxfertest"'\''' --storage-account-key ''\''"fjJswxR7u4564356457645754S/sjBsQDrBZVFuC+rDPlPW88zPbzGNHkuFn46ETlh5aPZZ3B2342352345dEEg=="'\''' --local-path /backup
example-mssql-blobxfer-db-backup  | Traceback (most recent call last):
example-mssql-blobxfer-db-backup  |   File "/usr/bin/blobxfer", line 8, in <module>
example-mssql-blobxfer-db-backup  |     sys.exit(cli())
example-mssql-blobxfer-db-backup  |              ^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
example-mssql-blobxfer-db-backup  |     return self.main(*args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1078, in main
example-mssql-blobxfer-db-backup  |     rv = self.invoke(ctx)
example-mssql-blobxfer-db-backup  |          ^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
example-mssql-blobxfer-db-backup  |     return _process_result(sub_ctx.command.invoke(sub_ctx))
example-mssql-blobxfer-db-backup  |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
example-mssql-blobxfer-db-backup  |     return ctx.invoke(self.callback, **ctx.params)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 783, in invoke
example-mssql-blobxfer-db-backup  |     return __callback(*args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/decorators.py", line 92, in new_func
example-mssql-blobxfer-db-backup  |     return ctx.invoke(f, obj, *args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/click/core.py", line 783, in invoke
example-mssql-blobxfer-db-backup  |     return __callback(*args, **kwargs)
example-mssql-blobxfer-db-backup  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer_cli/cli.py", line 1129, in upload
example-mssql-blobxfer-db-backup  |     ctx.initialize(settings.TransferAction.Upload)
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer_cli/cli.py", line 71, in initialize
example-mssql-blobxfer-db-backup  |     self.credentials = settings.create_azure_storage_credentials(
example-mssql-blobxfer-db-backup  |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer_cli/settings.py", line 425, in create_azure_storage_credentials
example-mssql-blobxfer-db-backup  |     creds.add_storage_account(name, key, endpoint)
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer/operations/azure/__init__.py", line 69, in add_storage_account
example-mssql-blobxfer-db-backup  |     self._storage_accounts[name] = StorageAccount(
example-mssql-blobxfer-db-backup  |                                    ^^^^^^^^^^^^^^^
example-mssql-blobxfer-db-backup  |   File "/usr/lib/python3.11/site-packages/blobxfer/operations/azure/__init__.py", line 136, in __init__
example-mssql-blobxfer-db-backup  |     raise ValueError(
example-mssql-blobxfer-db-backup  | ValueError: specified storage account key is invalid for storage account: '"blobxfertest"'
example-mssql-blobxfer-db-backup  | + move_exit_code=1
example-mssql-blobxfer-db-backup  | + '[' SHA1 '!=' none ']'
example-mssql-blobxfer-db-backup  | + run_as_user rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz.sha1
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz.sha1
example-mssql-blobxfer-db-backup  | + run_as_user rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + run_as_user rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + sudo -Eu dbbackup rm -rf /tmp/backups/mssql_test1_example-db-host_20231204-100143.bak.gz
example-mssql-blobxfer-db-backup  | + var_true TRUE
example-mssql-blobxfer-db-backup  | + '[' true = true ']'
example-mssql-blobxfer-db-backup  | + debug off
example-mssql-blobxfer-db-backup  | + case "${1}" in
example-mssql-blobxfer-db-backup  | + backup_job_log_level='notice}'
example-mssql-blobxfer-db-backup  | + CONTAINER_LOG_LEVEL=NOTICE
example-mssql-blobxfer-db-backup  | + DEBUG_MODE=FALSE
example-mssql-blobxfer-db-backup  | + SHOW_OUTPUT=
example-mssql-blobxfer-db-backup  | + var_true FALSE
example-mssql-blobxfer-db-backup  | + '[' false = true ']'
example-mssql-blobxfer-db-backup  | + '[' false = yes ']'
example-mssql-blobxfer-db-backup  | + set +x
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:45 [ERROR] ** [01-example-db-host__test1] Moving of backup 'mssql_test1_example-db-host_20231204-100143.bak.gz' reported errors
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:46 [NOTICE] ** [01-example-db-host__test1] DB Backup for 'test1' time taken: Hours: 0 Minutes: 00 Seconds: 03
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:46 [ERROR] ** [01-example-db-host__test1] Skipping Cleaning up old backups because there were errors in backing up
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:46 [INFO] ** [01-example-db-host__test1] Backup 01 routines finish time: 2023-12-04 10:01:46 PST with exit code 0
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:46 [NOTICE] ** [01-example-db-host__test1] Backup 01 routines time taken: Hours: 0 Minutes: 00 Seconds: 03
example-mssql-blobxfer-db-backup  | 2023-12-04.10:01:46 [NOTICE] ** [01-example-db-host__test1] Sleeping for another 86397 seconds. Waking up at 2023-12-05 10:01:43 PST

I took the value of the variables from here image

tiredofit commented 10 months ago

Remove the quotes surrounding your key and ID perhaps?

tiredofit commented 10 months ago

In the meantime to catch this in future i have added a safety check to not run blobxfer without the proper environment variables.

lazarevdmitriy commented 10 months ago

When I deleted quotas everything works well.

# cat compose.yml
.....
- DB01_BLOBXFER_STORAGE_ACCOUNT=blobxfertest
- DB01_BLOBXFER_STORAGE_ACCOUNT_KEY=qwertyu235253425iop12234234253245==
tiredofit commented 10 months ago

Fantastic! Thanks for helping out with this!

lazarevdmitriy commented 10 months ago

Tomorrow I will plan to run this tool in k8s.

lazarevdmitriy commented 10 months ago

Can we backup to Azure blob container instead of Azure file share?

tiredofit commented 10 months ago

That I'm not sure. I am not familiar with blobxfer/Azure. This was added upon request from a user..