search

Percona-Lab / mongodb_consistent_backup

A tool for performing consistent backups of MongoDB Clusters or Replica Sets
https://www.percona.com
Apache License 2.0
276 stars 81 forks source link

Cannot upload backup to google cloud storage #241

Open umsalotagi opened 6 years ago

umsalotagi commented 6 years ago

........................................................config

  upload:
  #  method: none
     method: gs
  #  remove_uploaded: [true|false] (default: false)
  #  retries: [1+]           (default: 5)
  #  threads: [1+]           (default: 4)
     gs:
       project_id: linen-radio-180413
       access_key: XXXXXXXXXXXXXXXXX
       secret_key: fgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6Q+
       bucket_name: xxxxxxx-database-bckup
       bucket_prefix:
  #  rsync:

also tried with config default bucket_prefix:

I got following error

[2017-11-11 15:36:18,957] [INFO] [MainProcess] [Stage:init:31] Notify stage disabled, skipping
[2017-11-11 15:36:21,238] [INFO] [MainProcess] [State:init:145] Initializing root state directory /var/lib/mongodb-consistent-backup/default
[2017-11-11 15:36:21,239] [INFO] [MainProcess] [State:load_backups:162] Found 2 existing completed backups for set
[2017-11-11 15:36:21,240] [INFO] [MainProcess] [State:init:131] Initializing backup state directory: /var/lib/mongodb-consistent-backup/default/20171111_1536
[2017-11-11 15:36:21,241] [ERROR] [MainProcess] [Main:exception:209] Problem starting uploader! Error: 'bucket'
Traceback (most recent call last):
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Main.py", line 251, in run
    self.backup_directory
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Upload.py", line 11, in __init__
    self.init()
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Pipeline/Stage.py", line 45, in init
    **self.args
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Gs/Gs.py", line 34, in __init__
    self.bucket          = self.config.upload.gs.bucket
  File "/var/lib/mongodb-consistent-backup/.pex/install/yconf-0.3.4-py2-none-any.whl.4c32acf64a3f2408d71fe4642e2daf2f77807381/yconf-0.3.4-py2-none-any.whl/yconf/util.py", line 52, in __getattr__
    raise AttributeError(e)
AttributeError: 'bucket'
[2017-11-11 15:36:21,242] [INFO] [MainProcess] [Main:cleanup_and_exit:163] Starting cleanup procedure! Stopping running threads
[2017-11-11 15:36:21,244] [INFO] [MainProcess] [Main:cleanup_and_exit:193] Cleanup complete, exiting
[2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:rotate:81] Running rotation of log files
[2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:compress:60] Compressing log file: /var/log/mongodb-consistent-backup/backup.default.20171111_1529.log

It says AttributeError: 'bucket' , there is not such attribute named bucket

dbmurphy commented 6 years ago

This looks like a versioning issue , do you have a GS Python module installed on you systems python?

Sent from my iPhone

On Nov 11, 2017, at 15:43, umsalotagi notifications@github.com wrote:

........................................................config

upload:

method: none

 method: gs

remove_uploaded: [true|false] (default: false)

retries: [1+] (default: 5)

threads: [1+] (default: 4)

 gs:
   project_id: linen-radio-180413
   access_key: XXXXXXXXXXXXXXXXX
   secret_key: fgxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6Q+
   bucket_name: xxxxxxx-database-bckup
   bucket_prefix:

rsync:

also tried with config default bucket_prefix:

I got following error

[2017-11-11 15:36:18,957] [INFO] [MainProcess] [Stage:init:31] Notify stage disabled, skipping [2017-11-11 15:36:21,238] [INFO] [MainProcess] [State:init:145] Initializing root state directory /var/lib/mongodb-consistent-backup/default [2017-11-11 15:36:21,239] [INFO] [MainProcess] [State:load_backups:162] Found 2 existing completed backups for set [2017-11-11 15:36:21,240] [INFO] [MainProcess] [State:init:131] Initializing backup state directory: /var/lib/mongodb-consistent-backup/default/20171111_1536 [2017-11-11 15:36:21,241] [ERROR] [MainProcess] [Main:exception:209] Problem starting uploader! Error: 'bucket' Traceback (most recent call last): File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Main.py", line 251, in run self.backup_directory File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Upload.py", line 11, in init self.init() File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Pipeline/Stage.py", line 45, in init **self.args File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.2.0-py2-none-any.whl.0ca97fff9d52c9fcb103096a08a99409e2177e0b/mongodb_consistent_backup-1.2.0-py2-none-any.whl/mongodb_consistent_backup/Upload/Gs/Gs.py", line 34, in init self.bucket = self.config.upload.gs.bucket File "/var/lib/mongodb-consistent-backup/.pex/install/yconf-0.3.4-py2-none-any.whl.4c32acf64a3f2408d71fe4642e2daf2f77807381/yconf-0.3.4-py2-none-any.whl/yconf/util.py", line 52, in getattr raise AttributeError(e) AttributeError: 'bucket' [2017-11-11 15:36:21,242] [INFO] [MainProcess] [Main:cleanup_and_exit:163] Starting cleanup procedure! Stopping running threads [2017-11-11 15:36:21,244] [INFO] [MainProcess] [Main:cleanup_and_exit:193] Cleanup complete, exiting [2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:rotate:81] Running rotation of log files [2017-11-11 15:36:21,245] [INFO] [MainProcess] [Logger:compress:60] Compressing log file: /var/log/mongodb-consistent-backup/backup.default.20171111_1529.log

It says AttributeError: 'bucket' , there is not such attribute named bucket

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

umsalotagi commented 6 years ago

I have run this script through docker by following command

docker run -it --net=host -v "/data/mongobackup/conf:/conf:Z" -v "/data/mongobackup/data:/var/lib/mongodb-consistent-backup:Z" -v "/data/mongobackup/logs:/var/log/mongodb-consistent-backup:Z" timvaillancourt/mongodb_consistent_backup:latest --config=/conf/mongodb-consistent-backup.conf