search

alexkravets / heroku-mongo-backup

Backup mongodb on Heroku and push it to S3 or FTP with cron task.
https://github.com/alexkravets/heroku-mongo-backup
55 stars 33 forks source link

Error thrown with MAX_BACKUPS option #13

Closed nicholasjhenry closed 5 years ago

nicholasjhenry commented 11 years ago

For example:

rake mongo:backup MAX_BACKUPS=7

results in the backtrace below. Also it seems to hang for a long time:

real    28m23.637s
user    0m0.642s
sys     0m0.207s
XML declaration allowed only at the start of the document (Nokogiri::XML::SyntaxError)                                                                                                                                                [2/1918]
/app/vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.5/lib/nokogiri/xml/sax/push_parser.rb:47:in `native_write'
/app/vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.5/lib/nokogiri/xml/sax/push_parser.rb:47:in `write'
/app/vendor/bundle/ruby/1.9.1/gems/fog-1.7.0/lib/fog/core/connection.rb:16:in `block in request'
/app/vendor/bundle/ruby/1.9.1/gems/excon-0.16.7/lib/excon/response.rb:44:in `call'
/app/vendor/bundle/ruby/1.9.1/gems/excon-0.16.7/lib/excon/response.rb:44:in `parse'
/app/vendor/bundle/ruby/1.9.1/gems/excon-0.16.7/lib/excon/connection.rb:275:in `request_kernel'
/app/vendor/bundle/ruby/1.9.1/gems/excon-0.16.7/lib/excon/connection.rb:103:in `request'
/app/vendor/bundle/ruby/1.9.1/gems/fog-1.7.0/lib/fog/core/connection.rb:20:in `request'
/app/vendor/bundle/ruby/1.9.1/gems/fog-1.7.0/lib/fog/aws/storage.rb:392:in `request'
/app/vendor/bundle/ruby/1.9.1/gems/fog-1.7.0/lib/fog/aws/requests/storage/get_bucket.rb:55:in `get_bucket'
/app/vendor/bundle/ruby/1.9.1/gems/fog-1.7.0/lib/fog/aws/models/storage/directories.rb:24:in `get'
/app/vendor/bundle/ruby/1.9.1/gems/fog-1.7.0/lib/fog/aws/models/storage/files.rb:32:in `all'
/app/vendor/bundle/ruby/1.9.1/gems/heroku-mongo-backup-0.4.3/lib/s3_helpers.rb:132:in `block in remove_old_backup_files'
/app/vendor/bundle/ruby/1.9.1/gems/heroku-mongo-backup-0.4.3/lib/s3_helpers.rb:132:in `each'
/app/vendor/bundle/ruby/1.9.1/gems/heroku-mongo-backup-0.4.3/lib/s3_helpers.rb:132:in `collect'
/app/vendor/bundle/ruby/1.9.1/gems/heroku-mongo-backup-0.4.3/lib/s3_helpers.rb:132:in `remove_old_backup_files'
/app/vendor/bundle/ruby/1.9.1/gems/heroku-mongo-backup-0.4.3/lib/heroku-mongo-backup.rb:218:in `backup'
/app/vendor/bundle/ruby/1.9.1/gems/heroku-mongo-backup-0.4.3/lib/tasks/heroku_mongo_backup.rake:8:in `block (2 levels) in <top (required)>'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `call'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `block in execute'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `each'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `execute'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/task.rb:158:in `block in invoke_with_call_chain'
/app/vendor/ruby-1.9.3/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/task.rb:151:in `invoke_with_call_chain'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/task.rb:144:in `invoke'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:116:in `invoke_task'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block (2 levels) in top_level'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `each'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block in top_level'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:88:in `top_level'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:66:in `block in run'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:63:in `run'
/app/vendor/bundle/ruby/1.9.1/gems/rake-0.9.2.2/bin/rake:33:in `<top (required)>'
/app/vendor/bundle/ruby/1.9.1/bin/rake:23:in `load'
/app/vendor/bundle/ruby/1.9.1/bin/rake:23:in `<main>'
alexkravets commented 11 years ago

Hi Nicholas, yeah I see this issue too. Will handle it soon. Still feel free to make a commit.

nicholasjhenry commented 11 years ago

FYI: This works OK on my development machine (OSX), the error is only repeatable on Heroku. I'm not sure if this is a timing out type of issues considering it was trying to delete all my objects (well it can only delete a max of 1000 since that's the cap on directories.files.all) in the bucket. For more info: https://github.com/alexkravets/heroku-mongo-backup/commit/ddaefab4143f7965bcb57a049c7a6a45938f8c4f#commitcomment-2316212.

alexkravets commented 11 years ago

Yes, same for me.

On Dec 17, 2012, at 1:37 AM, Nicholas Henry notifications@github.com wrote:

FYI: This works OK on my development machine (OSX), the error is only repeatable on Heroku.

— Reply to this email directly or view it on GitHub.