search

mattconnolly / rails-backup-migrate

A gem providing a rake task for backing up and migrating databases and files between rails instances.
Other
26 stars 16 forks source link

Character set conversion #4

Open toddh opened 13 years ago

toddh commented 13 years ago

When I tried it in a Ubuntu environment (admittedly hosted via VirtualBox in Windows 7 64-bit), I got::

rake aborted! “\xE2″ from ASCII-8BIT to UTF-8

I looked at my database.yml and they’re set to utf8 and the collation on the database tables is “utf8_unicode_ci.” Mysql is running in the Ubuntu machine.

mattconnolly commented 13 years ago

I'm setting up some tests at the moment, but everything I am trying is just working fine in UTF8. Can you try it with rake --trace so I can see where the exception is coming from? Where is the Ascii 8 bit string coming from?

Otherwise, I can't reproduce it and that'll make it much harder to help fix.

toddh commented 13 years ago

Here's an output from rake --trace. I think I mentioned this, but in case I didn't. I'm running in Ubuntu (VirtualBox inside of Windows 7) and the MySQL server is also in Ubuntu on the same machine. As near as I can tell, everything should be UTF8.

--Todd

/home/todd/.rvm/rubies/ruby-1.9.2-p290/bin/ruby -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) /home/todd/.rvm/gems/ruby-1.9.2-p290@global/bin/rake site:backup[backup_file] --trace * Invoke site:backup (first_time) * Invoke site:backup:_save_db_schema (first_time) * Invoke environment (first_time) * Execute environment * Invoke db:schema:dump (first_time) * Invoke db:load_config (first_time) * Invoke rails_env (first_time) * Execute rails_env * Execute db:load_config * Execute db:schema:dump * Execute site:backup:_save_db_schema * Invoke site:backup:_save_db_to_yml (first_time) * Invoke environment * Execute site:backup:_save_db_to_yml rake aborted! "\xE2" from ASCII-8BIT to UTF-8 /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:17:in write' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:17:in end_document' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:17:in visit_Psych_Nodes_Document' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:10:in accept' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in block in visit_Psych_Nodes_Stream' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in each' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/emitter.rb:10:in visit_Psych_Nodes_Stream' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/visitors/visitor.rb:11:in accept' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych/nodes/node.rb:36:in to_yaml' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/psych.rb:166:indump' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:104:in block (2 levels) in save_db_to_yml' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:104:in open' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:104:in block in save_db_to_yml' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:102:in each' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/rails-backup-migrate.rb:102:in save_db_to_yml' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rails-backup-migrate-0.0.8/lib/tasks/rails-backup-migrate.rake:25:in block (3 levels) in <top (required)>' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:205:in call' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:205:in block in execute' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:200:in each' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:200:in execute' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:158:in block in invoke_with_call_chain' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:151:in invoke_with_call_chain' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:176:in block in invoke_prerequisites' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:174:in each' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:174:in invoke_prerequisites' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:157:in block in invoke_with_call_chain' /home/todd/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:151:in invoke_with_call_chain' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/task.rb:144:in invoke' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:112:in invoke_task' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:90:in block (2 levels) in top_level' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:90:in each' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:90:in block in top_level' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:129:in standard_exception_handling' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:84:in top_level' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:62:in block in run' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:129:in standard_exception_handling' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/application.rb:59:in run' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/bin/rake:32:in <top (required)>' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/bin/rake:19:in load' /home/todd/.rvm/gems/ruby-1.9.2-p290@global/bin/rake:19:in<top (required)>' -e:1:in load' -e:1:in

' Tasks: TOP => site:backup => site:backup:_save_db_to_yml

Process finished with exit code 1

On Tue, Aug 9, 2011 at 7:06 AM, soundevolution < reply@reply.github.com>wrote:

I'm setting up some tests at the moment, but everything I am trying is just working fine in UTF8. Can you try it with rake --trace so I can see where the exception is coming from? Where is the Ascii 8 bit string coming from?

Otherwise, I can't reproduce it and that'll make it much harder to help fix.

Reply to this email directly or view it on GitHub:

https://github.com/soundevolution/rails-backup-migrate/issues/4#issuecomment-1762460

############################################ # Please note new email address: tahitt@gmail.com # (I'm migrating away from Yahoo mail)

toddh commented 13 years ago

I see I didn't ask your second question. Not being character set savvy, I'm not sure how to figure out where the Ascii 8 bit string is coming from. Is there a query I can run on the DB? I could perhaps do mysqldump and send that. Would that provide a clue?

mattconnolly commented 13 years ago

The stack trace helps. I can see it's happening in the yaml export in ruby 1.9.2. Admittedly, I'd only tested on 1.8.7 since the rails apps I've used all used that. It's easy to repro on ruby 1.9.2 with rails 3, and I'm taking a look into it.

mattconnolly commented 13 years ago

Ruby 1.9.2 definitely treats YML files very differently to 1.8.7. I have a solution not too far away...

mattconnolly commented 13 years ago

After chasing my tail round in circles, I undid a whole bunch of my changes, and then with only some minor changes, couldn't reproduce the issue in 1.9.2. I've updated version 0.0.9 with these minimal changes. Let me know if this helps or not...

toddh commented 13 years ago

I think that this does solve the problem -- but can't completely verify it because I'm running into potentially another issue (remember I'm a noob at Ruby):

It fails to create the tar file because files_to_archive is empty in save_db_to_yml. On my system, the culprit appears to be add_to_archive which only adds something to list if it starts with RAILS_ROOT (which I think is deprecate in 3.1). The string being passed in is "db/schema.rb" and "db/backup", neither of which starts with RAILS_ROOT.

I added an:

      else
        @files_to_archive << path

to add_to_archive and that seems to have fixed the problem.

Thoughts?

The tar file does look correct, but I haven't had time to try a restore yet.