invalid byte sequence in UTF-8 in Rack::Cache::Key

[tc]

I'm seeing a lot of these errors in our logs:

cache error: invalid byte sequence in UTF-8
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:46:in `split'
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:46:in `block in query_string'
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:45:in `map'
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:45:in `query_string'

    def query_string
      return nil if @request.query_string.nil?

      @request.query_string.split(/[&;] */n).
46:        map { |p| unescape(p).split('=', 2) }.
        sort.
        map { |k,v| "#{escape(k)}=#{escape(v)}" }.
        join('&')
    end

[anderslemke]

I'm getting the same errors on our logs on Heroku (Cedar stack).

[barttenbrinke]

Removing the my entire bundle & cache fixed this for me.

[henrik]

I'm seeing this as well. Looks like it's possibly for any request with URL encoded Latin-1 in it, e.g.

cache error: invalid byte sequence in UTF-8
…
cache: [GET /kalix?q=sn%F6slunga] pass

[henrik]

Okay, I ended up doing this for now. Admittedly a kludge (and only for Ruby 1.9):

# Make Rack::Cache not break on Latin-1 query params.
# https://github.com/rtomayko/rack-cache/issues/47
class Rack::Cache::Key
  def unescape(x)
    super(x).encode("UTF-8", "ISO8859-1")
  end

  def escape(x)
    super(x.encode("ISO8859-1", "UTF-8")).encode("UTF-8", "ISO8859-1")
  end
end

In my case, I can rely on (I hope) query parameters to always be Latin-1 (ISO 8859-1), so this solution should work. Shouldn't be that hard to come up with a general solution, though – may give it a go.

[joe1chen]

Anyone have a general solution for this problem? We're also seeing this issue.

[jamsi]

Am also seeing this in our unicorn logs. Very strange.

[jun1st]

See same error a lot in the unicorn log file.

[xentek]

I feel like I've seen this, but not in a while. Have you tried adding the # encoding: utf-8 magic comment to the top of your config.ru or other relevant files?