search

mynewsdesk / deprecated-translate

A Ruby on Rails plugin with a web interface for translating I18n texts
http://developer.newsdesk.se/2009/01/21/translate-new-rails-i18n-plugin-with-a-nice-web-ui
MIT License
346 stars 87 forks source link

invalid byte sequence in UTF-8 #7

Open dbarison opened 14 years ago

dbarison commented 14 years ago

problem with rails 2.3.8 and ruby 1.9.1

mclark commented 13 years ago

It looks like I'm having the same issue with rails 2.3.8 and ruby 1.9.2.pre2. Here is the stack trace to reproduce:

invalid byte sequence in UTF-8

/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:135:in `scan'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:135:in `block in extract_files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:134:in `each'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:134:in `inject'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:134:in `extract_files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:15:in `files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:6:in `files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate_controller.rb:37:in `initialize_keys'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate_controller.rb:12:in `index'
/home/mclark/.rvm/gems/ruby-1.9.2-head/gems/actionpack-2.3.8/lib/action_controller/base.rb:1331:in `perform_action'
/home/mclark/.rvm/gems/ruby-1.9.2-head/gems/actionpack-2.3.8/lib/action_controller/filters.rb:617:in `call_filters'
mclark commented 13 years ago

For the record, this was caused by most source files in my app being in iso-8859-1 encoding, while the extract_files method loads the file assuming it is in utf-8 by default (not sure where this comes from, it could be a system default. I've fixed the issue locally by just changing the line to the following. I don't know enough about this stuff to make a proper fix, but this should do for me, for now.

IO.read(file, :encoding => 'iso-8859-1' ).scan(i18n_lookup_pattern).flatten.map(&:to_sym).each do |key|