Posted by guest at 2006-09-05 08:21:36
when i get messages in russian from qip icq clients they are not encoded right,
i get only strange letters. the other side can read my russian fine
how can i help to fix this problem? need some logged message or something?
i love to chat to icq users using pyicqt so it would eb really nice to fix this
Posted by guest at 2006-09-05 08:53:22
i logged one of those messages can you see a problem here? :)
[2006-09-05 10:36:08] user = <OSCARUser 288133356, flags ['free', 'away',
'icq'], online for 342 minutes, idle for 343 minutes>, data =
000200190501000201010101000f00000000cfeee6e0ebf3e9f1f2e020000b0000001300010f
[2006-09-05 10:36:08] tlvs = {19: '\x0f', 2:
'\x05\x01\x00\x02\x01\x01\x01\x01\x00\x0f\x00\x00\x00\x00\xcf\xee\xe6\xe0\xeb\xf
3\xe9\xf1\xf2\xe0 ', 11: ''}
[2006-09-05 10:36:08] unknown TLV for incoming IM, 0013, '\x0f'
[2006-09-05 10:36:08] Message data:
'\x05\x01\x00\x02\x01\x01\x01\x01\x00\x0f\x00\x00\x00\x00\xcf\xee\xe6\xe0\xeb\xf
3\xe9\xf1\xf2\xe0 '
[2006-09-05 10:36:08] INFO :: admin@jabber.cheatpoint.de :: icqt.B ::
receiveMessage :: {'multiparts': [('\xcf\xee\xe6\xe0\xeb\xf3\xe9\xf1\xf2\xe0 ',
'ascii', 'standard')], 'flags': ['typingnot'], 'user': <OSCARUser 288133356,
flags ['free', 'away', 'icq'], online for 342 minutes, idle for 343 minutes>,
'icq2jid': <function icq2jid at 0xb70a0ed4>} :: 288133356
[('\xcf\xee\xe6\xe0\xeb\xf3\xe9\xf1\xf2\xe0 ', 'ascii', 'standard')]
['typingnot']
[2006-09-05 10:36:08] INFO :: admin@jabber.cheatpoint.de :: icqt.B ::
receiveMessage :: {'sourcejid': '288133356@icq.cheatpoint.de', 'encoding':
'iso-8859-1', 'text': '\xcf\xee\xe6\xe0\xeb\xf3\xe9\xf1\xf2\xe0 ',
'multiparts':
[('\xcf\xee\xe6\xe0\xeb\xf3\xe9\xf1\xf2\xe0 ', 'ascii', 'standard')], 'flags':
['typingnot'], 'user': <OSCARUser 288133356, flags ['free', 'away', 'icq'],
online for 342 minutes, idle for 343 minutes>, 'icq2jid': <function
icq2jid at 0xb70a0ed4>} :: Using encoding iso-8859-1
[2006-09-05 10:36:08] INFO :: :: :: prepxhtml :: {'s':
'\xc3\x8f\xc3\xae\xc3\xa6\xc3\xa0\xc3\xab\xc3\xb3\xc3\xa9\xc3\xb1\xc3\xb2\xc3\xa
0 '} :: Got
'\xc3\x8f\xc3\xae\xc3\xa6\xc3\xa0\xc3\xab\xc3\xb3\xc3\xa9\xc3\xb1\xc3\xb2\xc3\xa
0 '
[2006-09-05 10:36:08] INFO :: :: :: prepxhtml :: {'all_regex':
<_sre.SRE_Pattern object at 0xb70b3ec0>, 'font_regex':
<_sre.SRE_Pattern object at 0xb7044188>, 's':
'\xc3\x8f\xc3\xae\xc3\xa6\xc3\xa0\xc3\xab\xc3\xb3\xc3\xa9\xc3\xb1\xc3\xb2\xc3\xa
0 '} :: Failed
[2006-09-05 10:36:08] INFO :: admin@jabber.cheatpoint.de :: Session ::
sendMessage :: {'body': u'\xcf\xee\xe6\xe0\xeb\xf3\xe9\xf1\xf2\xe0', 'delay':
None, 'fro': '288133356@icq.cheatpoint.de', 'mtype': 'chat', 'to':
u'admin@jabber.cheatpoint.de', 'xhtml': None, 'nickname': None} ::
[2006-09-05 10:36:08] INFO :: :: :: sendMessage :: {'body':
u'\xcf\xee\xe6\xe0\xeb\xf3\xe9\xf1\xf2\xe0', 'delay': None, 'fro':
'288133356@icq.cheatpoint.de', 'mtype': 'chat', 'to':
u'admin@jabber.cheatpoint.de', 'xhtml': None, 'nickname': None, 'pytrans':
<main.PyTransport instance at 0xb70bb0ac>} ::
[2006-09-05 10:36:15] INFO :: admin@jabber.cheatpoint.de :: Session ::
onMessage
:: {'el': <twisted.words.xish.domish.Element object at 0xaa77b4cc>} ::
[2006-09-05 10:36:15] INFO :: admin@jabber.cheatpoint.de :: Session ::
onMessage
:: {'body': '', 'el': <twisted.words.xish.domish.Element object at
0xaa77b4cc>, 'chatStates': None, 'froj':
<twisted.words.protocols.jabber.jid.JID instance at 0xb7033b6c>, 'fro':
u'admin@jabber.cheatpoint.de/Notebook', 'child':
<twisted.words.xish.domish.Element object at 0xaa77b5cc>, 'composing':
True, 'mID': u'a2748a', 'deepchild': <twisted.words.xish.domish.Element
object at 0xaa77b0ec>, 'mtype': None, 'to': u'288133356@icq.cheatpoint.de',
'noerror': False, 'error': None, 'xhtml': None, 'messageEvent': True,
'chatStateEvent': None, 'autoResponse': 0, 'toj':
<twisted.words.protocols.jabber.jid.JID instance at 0xb7033f2c>} :: Typing
notification True
Posted by guest at 2006-09-05 11:02:56
i found out more
the clients send the message encoded in Windows-1251 encoding
(often called CP-1251 where CP stands for Code Page) or KOI8-R
both are resulting in being displayed as iso-8859-1 and thus unreadable
i hope this helps
with miranda it works fine as it sends in unicode
Posted by jadestorm at 2006-09-05 19:41:17
Do you have config.xml set to cp-1251? (<encoding> I think it is)
Posted by guest at 2006-09-07 17:26:32
i have <encoding>iso-8859-1</encoding>
i tried cp-1251 then this is the result :
ICQ connection lost! Reason: [Failure instance: Traceback (failure with no
frames): exceptions.LookupError: unknown encoding: cp-1251
]
(i received a message but instead the message got this)
i need iso-8859-1 for all my german contacts anyway :(
Posted by guest at 2006-09-07 17:28:35
and in logfile of the gateway :
Traceback (most recent call last):
File "/usr/local/lib/python2.4/site-packages/twisted/python/log.py",
line 53, in callWithLogger
return callWithContext({"system": lp}, func, *args, **kw)
File "/usr/local/lib/python2.4/site-packages/twisted/python/log.py",
line 38, in callWithContext
return context.call({ILogContext: newCtx}, func, *args, **kw)
File
"/usr/local/lib/python2.4/site-packages/twisted/python/context.py",
line 59, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File
"/usr/local/lib/python2.4/site-packages/twisted/python/context.py",
line 37, in callWithContext
return func(*args,**kw)
--- <exception caught here> ---
File
"/usr/local/lib/python2.4/site-packages/twisted/internet/selectreactor.py&q
uot;, line 139, in _doReadOrWrite
why = getattr(selectable, method)()
File
"/usr/local/lib/python2.4/site-packages/twisted/internet/tcp.py", line
348, in doRead
return self.protocol.dataReceived(data)
File "/home/jabber/icq_gateway/src/tlib/oscar.py", line 642, in
dataReceived
state=func(flap)
File "/home/jabber/icq_gateway/src/tlib/oscar.py", line 759, in
oscar_Data
func(snac)
File "/home/jabber/icq_gateway/src/tlib/oscar.py", line 1327, in
oscar_04_07
self.receiveMessage(user, multiparts, flags)
File "/home/jabber/icq_gateway/src/legacy/icqt.py", line 219, in
receiveMessage
text = text.decode(encoding, "replace")
exceptions.LookupError: unknown encoding: cp-1251
icq gateway connection was closed after this
Posted by jadestorm at 2006-09-08 19:17:34
Maybe it's cp1251? I can't imagine why the encoding wouldn't exist at all.
You definitely can -not- support two encodings at the same time. the iso/cp
encodings suck like that. There's no way to look at them programmatically and
tell what they are. Ideally the solution is to go with unicode, but not
everyone has ICQ clients that do unicode. [if they did, this wouldn't have
come
up, because i do support unicode]
Posted by guest at 2006-09-11 07:28:33
i tested it and it worked, BUT :)
now all german umlauts are russian characters which is bad
could you mb make something where i could for example have an xml file where i
would add all the jabber ids or icq uins of the qip users so it would use
cp1251
for them and for all others the default from config.xml or unicode ?
this would really rock because i cant convince the qip users to use miranda
unicode or some other icq client that uses unicode :(
Posted by guest at 2006-09-11 08:29:05
maybe something like this
<contact>
<jabberid>34298754@icq.cheatpoint.de</jabberid>
<encoding>cp1251</encoding>
</contact>
and if the contact isnt found there it will use the standards as we have now
so i think this could be global for all gateway users
Posted by jadestorm at 2006-09-11 12:22:31
Hrm. That could get very complex. Most of the encoding handling is done
within
oscar.py but oscar.py is not supposed to know anything about config.xml. Also,
putting it in config.xml is not very scalable... if you have a lot of users
making use of this, you would be editing the config file for all of them.
(eww)
Sounds like yet another thing that would go well in the user preferences
functionality that I still plan to work on. =D
Posted by jadestorm at 2006-09-11 12:33:09
What a wonderful suggestion this is. ;D Never dawned on me!
[note that this was a bug report]
Original issue reported on code.google.com by jadest...@gmail.com on 15 Dec 2007 at 5:29
Original issue reported on code.google.com by
jadest...@gmail.comon 15 Dec 2007 at 5:29