Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8

[dbkbali]

When migrating from a legacy database to mongodb using mongoid 3 moped is generating the above error message:

Connecting to {:adapter=>"mysql", :host=>"127.0.0.1", :port=>"3306", :username=>"root", :password=>"", :database=>"w2d_dev", :encoding=>"utf8"}
LV pairs -->>>>>>>     {"id"=>"Makanan", "ru"=>"Пища", "zh"=>"食品", "jp"=>"食品", "ko"=>"음식", "fr"=>"Gastronomie", "de"=>"Nahrungsmittel", "en"=>"Food"}

<Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8>
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/regexp.rb:29:in `__bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:37:in `block in __bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:36:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:36:in `__bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/array.rb:32:in `block in __bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/array.rb:31:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/array.rb:31:in `each_with_index'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/array.rb:31:in `__bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:37:in `block in __bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:36:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:36:in `__bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:37:in `block in __bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:36:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/extensions/hash.rb:36:in `__bson_dump__'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/bson/document.rb:11:in `serialize'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/protocol/message.rb:136:in `serialize_selector'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/protocol/message.rb:296:in `block (2 levels) in serialize'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/protocol/message.rb:295:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/protocol/message.rb:295:in `block in serialize'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/protocol/message.rb:292:in `tap'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/protocol/message.rb:292:in `serialize'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/connection.rb:137:in `block in write'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/connection.rb:135:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/connection.rb:135:in `write'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:489:in `block (2 levels) in flush'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:113:in `ensure_connected'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:488:in `block in flush'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:503:in `logging'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:487:in `flush'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:476:in `process'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:70:in `command'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/session/context.rb:39:in `block in command'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/session/context.rb:97:in `block in with_node'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/cluster.rb:135:in `block in with_primary'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/node.rb:158:in `ensure_primary'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/cluster.rb:134:in `with_primary'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/session/context.rb:96:in `with_node'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/session/context.rb:38:in `command'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/database.rb:83:in `command'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/moped-1.0.0.rc/lib/moped/query.rb:36:in `count'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/contextual/mongo.rb:56:in `count'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/contextual.rb:18:in `count'
  /Users/dbk/Development/w2d/app/models/unique_in_locale_validator.rb:13:in `search_locale'
  /Users/dbk/Development/w2d/app/models/unique_in_locale_validator.rb:33:in `block in unique_in_locale'
  /Users/dbk/Development/w2d/app/models/unique_in_locale_validator.rb:32:in `each'
  /Users/dbk/Development/w2d/app/models/unique_in_locale_validator.rb:32:in `unique_in_locale'
  /Users/dbk/Development/w2d/app/models/unique_in_locale_validator.rb:3:in `validate_each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validator.rb:153:in `block in validate'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validator.rb:150:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validator.rb:150:in `validate'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:310:in `_callback_before_191'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:440:in `_run__3683343782699780073__validate__85299118211278988__callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:405:in `__run_callback'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:385:in `_run_validate_callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:81:in `run_callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/callbacks.rb:95:in `run_callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validations.rb:212:in `run_validations!'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validations/callbacks.rb:53:in `block in run_validations!'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:447:in `_run__3683343782699780073__validation__85299118211278988__callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:405:in `__run_callback'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:385:in `_run_validation_callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/callbacks.rb:81:in `run_callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/callbacks.rb:95:in `run_callbacks'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validations/callbacks.rb:53:in `run_validations!'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validations.rb:179:in `valid?'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/validations.rb:77:in `valid?'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activemodel-3.2.3/lib/active_model/validations.rb:187:in `invalid?'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/persistence/insertion.rb:22:in `prepare'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/persistence/operations/insert.rb:26:in `persist'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/persistence.rb:49:in `insert'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/persistence.rb:188:in `upsert'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/persistence.rb:75:in `save!'
  /Users/dbk/Development/w2d/db/seed/create_classification_collection.rb:38:in `block in <top (required)>'
  /Users/dbk/Development/w2d/db/seed/create_classification_collection.rb:29:in `each'
  /Users/dbk/Development/w2d/db/seed/create_classification_collection.rb:29:in `<top (required)>'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:251:in `require'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:251:in `block in require'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:236:in `load_dependency'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:251:in `require'
  /Users/dbk/Development/w2d/db/seeds.rb:35:in `<top (required)>'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:245:in `load'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:245:in `block in load'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:236:in `load_dependency'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/activesupport-3.2.3/lib/active_support/dependencies.rb:245:in `load'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@rails32/gems/mongoid-3.0.0.rc/lib/mongoid/railties/database.rake:13:in `block (2 levels) in <top (required)>'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `call'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `block in execute'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `execute'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/task.rb:158:in `block in invoke_with_call_chain'
  /Users/dbk/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/task.rb:151:in `invoke_with_call_chain'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/task.rb:144:in `invoke'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:116:in `invoke_task'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block (2 levels) in top_level'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `each'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block in top_level'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:88:in `top_level'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:66:in `block in run'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/application.rb:63:in `run'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/bin/rake:33:in `<top (required)>'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/bin/rake:19:in `load'
  /Users/dbk/.rvm/gems/ruby-1.9.3-p194@global/bin/rake:19:in `<main>'
rake aborted!
incompatible character encodings: ASCII-8BIT and UTF-8

It is crashing out on validation when searching the database by regexp. Below is the validator:

class UniqueInLocaleValidator < ActiveModel::EachValidator
  def validate_each(record, attribute, value)
    (record.errors[attribute] << "#{record.class.to_s} is not unique in locale #{@already_in_locales} - please use another name") if !unique_in_locale(record, attribute)
  end

private
  def search_locale(record, attribute, locale, siblings)
    if !record.send("#{attribute.to_s}_translations")[locale].blank?
      pattern_locale = record.send("#{attribute.to_s}_translations")[locale]
      pattern = /\A(#{pattern_locale})\z/i
      checkrecord = record.class.all_of("#{attribute.to_s}.#{locale}" => pattern)
      checkrecord = checkrecord.and(siblings) if !siblings.empty?
      rcount = checkrecord.count

      if rcount == 0
        return false
      elsif rcount == 1 && record.persisted? && record.id == checkrecord.first.id
        return false
      else
        return true
      end
    end  
  end

  def unique_in_locale(record, attribute)
    @already_in_locales = ""
    siblings = {}
    siblings.merge!(:parent_id => record.parent_id) if record.respond_to?("parent_id")
    siblings.merge!(:level => record.level) if record.respond_to?("level")
    if record.fields[attribute.to_s].options[:localize] == true
      start_locale = I18n.locale
      record[attribute].keys.each do |locale|
        @already_in_locales = ("#{I18n.locale.to_s}" + " ") if search_locale(record, attribute,locale, siblings) 
      end 
      I18n.locale = start_locale
    end
    if @already_in_locales.blank? 
      return true
    else
      return false
    end
  end
end

[max-power]

got the same issue while doing a regexp search with umlauts

Collection.where(name: /hö/i)