utf8 mode still with problems

[GoogleCodeExporter]

What steps will reproduce the problem?
1. have utf8 mode active
2. connect to the mud using simplemu
3. try to type an accentuated character, you log out instead.

Original issue reported on code.google.com by mindboos...@gmail.com on 1 Apr 2011 at 1:44

[GoogleCodeExporter]

I am not familiar with simplemu. Is that the only client that has a problem? 
Are you sure it is set to utf-8 encoding?

What message do you get in the error log?

Original comment by miss...@hotmail.com on 2 Apr 2011 at 6:22

• Changed state: Accepted

[GoogleCodeExporter]

> I am not familiar with simplemu.

Neither am I... I just know it is a client for Windows (some considering it the 
best). I doubt it is the only client having the issue (it would be really 
strange, specially since it doesn't have character issues with other MU*s), but 
it is the only case I've heard complaints about. 

> Are you sure it is set to utf-8 encoding?

No, the application doesn't seem to have anywhere where to configure the 
charset. Anyway, even if it was using the wrong utf-8, it shouldn't have this 
"reaction", it should just show an weird character, right?

> What message do you get in the error log?

2011-04-01 14:41:20+0100 [TZ,32,81.92.221.157] player P2 logged in
2011-04-01 14:41:25+0100 [TZ,32,81.92.221.157] Unhandled Error
        Traceback (most recent call last):
          File "/usr/lib/python2.5/site-packages/twisted/python/log.py", line 84, in callWithLogger
            return callWithContext({"system": lp}, func, *args, **kw)
          File "/usr/lib/python2.5/site-packages/twisted/python/log.py", line 69, in callWithContext
            return context.call({ILogContext: newCtx}, func, *args, **kw)
          File "/usr/lib/python2.5/site-packages/twisted/python/context.py", line 59, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/usr/lib/python2.5/site-packages/twisted/python/context.py", line 37, in callWithContext
            return func(*args,**kw)
        --- <exception caught here> ---
          File "/usr/lib/python2.5/site-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite
            why = getattr(selectable, method)()
          File "/usr/lib/python2.5/site-packages/twisted/internet/tcp.py", line 460, in doRead
            return self.protocol.dataReceived(data)
          File "/usr/lib/python2.5/site-packages/twisted/protocols/basic.py", line 251, in dataReceived
            why = self.lineReceived(line)
          File "/tzmud-read-only/src/tzprotocol.py", line 248, in lineReceived
            line = line.decode('utf-8')
          File "/usr/lib/python2.5/encodings/utf_8.py", line 16, in decode
            return codecs.utf_8_decode(input, errors, True)
        exceptions.UnicodeDecodeError: 'utf8' codec can't decode bytes in position 2-4: invalid data

2011-04-01 14:41:25+0100 [TZ,32,81.92.221.157] Lost a client!

Original comment by noori.re...@gmail.com on 4 Apr 2011 at 4:38

[GoogleCodeExporter]

For now you should at least apply this patch...

--- src/tzprotocol.py   2011-11-08 18:51:13.000000000 +0000
+++ src/tzprotocol.py-fix       2011-11-08 19:07:16.000000000 +0000
@@ -246,7 +246,7 @@

         try:
             line = line.decode('utf-8')
-        except UnicodeEncodeError:
+        except UnicodeDecodeError:
             print 'Cannot decode as utf-8'

         line = line.strip()

Original comment by mindboos...@gmail.com on 8 Nov 2011 at 7:08

[GoogleCodeExporter]

Makes sense. Applied.

Does this allow simplemenu to connect?

Original comment by miss...@hotmail.com on 8 Nov 2011 at 7:31

[GoogleCodeExporter]

I'm not sure, I never had the actual reproduction environment... But I would 
think that it probably does, and this fix can be closed without fuss (even if 
it might happen that it needs to be reopened in the future).

Also, thanks for the really quick response time ;-)

Original comment by mindboos...@gmail.com on 8 Nov 2011 at 8:02