informatikr / hedis

A Redis client library for Haskell.
http://hackage.haskell.org/package/hedis
BSD 3-Clause "New" or "Revised" License
329 stars 127 forks source link

Hedis returns escaped UTF8 escape sequences #24

Closed natesymer closed 9 years ago

natesymer commented 9 years ago

I recently had the pleasure of using Hedis in a project I was working on (I can't say details about the project). It was a great, clean library that was an absolute pleasure to use, except in one area: encoding. At one point, I was BRPOPing some text with the character é. Surprisingly enough, Hedis spat out a response with some messed up encoding, opting to return a UTF8 encoded value.

Upon farther investigation, the UTF8 encoding was done in such a way that the escape sequences were escaped themselves. I did some digging (including running redis-cli in raw mode) and I discovered that Redis properly returns irregular characters like é. Furthermore, it is well known that Redis doesn't modify/encode/decode/whatever data it receives.

So I wrote my own stripped down Redis client using Network.Socket.ByteString, and sure enough (in addition to being ever so slightly faster :p) the é character was returned correctly (I wrote the result to a file using Data.Text.IO and Data.Text.Encoding and use the UNIX cat command to read it).

Would a possible solution be to use Network.Socket.ByteString in place of whatever is being used currently?

informatikr commented 9 years ago

I think in the cases where the é is escaped you are using the Show instance of ByteString, e.g. by using print. The code below prints the escaped version in the first line and the é character in the second line.

{-# LANGUAGE OverloadedStrings #-}
module Main where

import qualified Data.ByteString.Char8 as B
import           Database.Redis

main :: IO ()
main = do
    conn <- connect defaultConnectInfo
    Right (Just val) <- runRedis conn (lindex "key" 0)

    print val
    B.putStrLn val
natesymer commented 9 years ago

Yes, I literally just realized this, the issues were coming from elsewhere. Thanks for the illustrative example! Sorry for the confusion.