Can't handle "Error reading from XML stream." - Githubissues

fritzy / SleekXMPP

Python 2.6+/3.1+ XMPP Library

http://groups.google.com/group/sleekxmpp-discussion

Other

1.1k stars 299 forks source link

Can't handle "Error reading from XML stream." #102

Closed ayanamist closed 12 years ago

ayanamist commented 13 years ago

My program use sleekxmpp 1.0RC2, and set auto_reconnect=True, After a long time running, it throws tons of message as following in my stdout file.

09-15 14:33:36 sleekxmpp.xmlstream.xmlstream ERROR Error reading from XML stream. 09-15 14:33:36 sleekxmpp.xmlstream.xmlstream DEBUG Ending read XML loop

Yes, tons of messages, 700KB in just two minutes, and i shutdown the program and restart. Everything works fine again. I know may be a bug there (sorry i haven't capture it), now i want to know how can i handle this situation when this happends again?

ayanamist commented 13 years ago

I have seen xmlsteam.py, found when SyntaxError, it just log it and let loop continue. Shall it reconnect when lots of SyntaxError?

fritzy commented 13 years ago

SyntaxError should cause a reconnect. I'll try to reproduce this.

ayanamist commented 13 years ago

No, it won't. Only KeyboardInterrupt, SystemExit, Socket.error will self.stop.set() which will let it reconnect. ( line 1088 - 1100 ). In __read_xml, when line 1149 capture a SyntaxError, it just log.error and return back to _process meanwhile _process won't capture SyntaxError again (Shall this be raised again?). Now self.stop.is_set() is still False, so the loop goes on, and so more SyntaxError. I think one or two SyntaxError is not the reason to reconnect, but more are dangerous. Shall it count and store the continuous SyntaxError and reconnect when there are too many of them?

fritzy commented 13 years ago

Technically, if we have an error parsing the XML from the server, we MUST send a stream error and disconnect in XMPP. Which server are you using?

ayanamist commented 13 years ago

GTalk server in DotCloud ( it seems DotCloud is on the Amazon Web Services). My program is here: https://code.google.com/p/twiotaku/

fritzy commented 13 years ago

I'll get this fixed tonight in the develop branch.

ayanamist commented 13 years ago

I hope this can be merged into 1.0RCx version or master branch. I stick to "stable" branch.

minhnhb commented 13 years ago

I have the same issue. Looking forward to see the fix :|

minhnhb commented 13 years ago

@Fritzy: Do you plan to use lxml.etree instead of celementtree?

fritzy commented 13 years ago

This is fixed in develop. Develop is simply stability fixes on RC2 right now, until 1.0-final is out. Please try it.

@minhnhb As far as lxml.etree, it'd be great to support it as an alternative. We're looking into it.

minhnhb commented 13 years ago

It reconnects the stream when seeing the error, but I'm sure to send a valid xml over the stream. Could it be in trouble with unicode strings?

minhnhb commented 13 years ago

I remove accents from xml when sending packages over xmlstream and it is working fine now. I'm sure that the problem is related to unicode strings.

fritzy commented 13 years ago

I'll continue to test then. We have unicode tests, and so I'm not sure why they'd break the XML parser. 1.0 may need to be held up until we can reproduce and fix.

In general, we HAVE to connect on a Syntax error, both in practicality, and in keeping with the RFCs. If sending unicode can reproduce it, then we'll have to find a way to fix it.

Are you sending unicode characters as part of the syntax of the XML? Can you should me a debug log that shows data causing the problem?

minhnhb commented 13 years ago

No, the xml data only contains unicode characters in its attributes and text.

You can try this:

<iq from="" to="" type=""> <query xmlns="some_ns"> <text> Game chia các phòng chơi theo tiêu chí cược, phòng cược 1$, 5$, 10$, 50$, 100$, 200$, 500$, 1k$, 5k$, 20k$, 100k$, 500k$ Người chơi chọn mức tiền cược nào sẽ thấy các phòng chơi tương ứng. Game sẽ bắt đầu sau một khoảng thời gian nhất định (10s) và khi có đủ người chơi (ít nhất là 2 người) Khi game bắt đầu, game sẽ đưa ra câu hỏi phân loại để sắp xếp vị trí ngồi (có 6 vị trí ngồi). Người nào trả lời gần đáp án nhất sẽ được chọn vị trí trước, gần thứ 2 được chọn vị trí 2, tương tự người trả lời gần đáp án thứ 3, 4, 5. Người trả lời xa đáp án nhất bắt buộc ngồi vị trí còn lại. Trong trường hợp chỉ có một người trả lời câu hỏi phân loại, thì game sẽ kết thúc ngay và người đó là người chiến thắng. Nếu có nhiều hơn hai người trả lời câu hỏi phân loại, sau khi sắp xếp ghế ngồi, game sẽ bắt đầu gửi cho người ở vị trí ghế nóng các câu hỏi, và người chơi buộc phải trả lời ít nhất một câu hỏi rồi mới có thể chuyển, nếu không sẽ bị loại. Khi người chơi bị loại thì sẽ quay trở lại với vai trò khán giả. Nếu người chơi chuyển thì người ở vị trí ghế ngay sau sẽ lên vị trí ghế nóng và bắt đầu trả lời câu hỏi. Người này phải trả lời câu hỏi ngay trước đó, và đến câu tiếp theo mới có quyền chuyển. Người nào ở lại cuối cùng hoặc trả lời hết 15 câu hỏi sẽ là người chiến thắng. Người chiến thắng sẽ được tất cả số tiền cược trong ván chơi. </text> </query> </iq>

I'm running ubuntu server 10.04 with python 2.6

minhnhb commented 13 years ago

I use stanza to set the unicode text.

ayanamist commented 13 years ago

I think you should use encode('UTF8') first. I always do this before post data to any libraries.

minhnhb commented 13 years ago

Take a look at the code (xmlstream.py), it encode('utf-8') data before sending to socket.

legastero commented 13 years ago

@minhnhb: Which XMPP server are you using? I just tested the example you gave with Prosody 0.8 and GTalk using Python 2.6/2.7, and no errors are occurring.

minhnhb commented 13 years ago

@legastero: We are running OpenFire 3.7.0. The xml above is just an example. There are tons of packages like that were exchanged between client and server. The server run for awhile and then throw the Syntax Error. If i remove accents from xml before sending, it works fine.

legastero commented 13 years ago

Ok, I think I found the underlying error here. The problem was a regression handling when the server terminates the stream. The processing loop was continuing to call __read_xml, which would cause SyntaxError to be raised because it can't find any starting element. That seems likely to be the situation you're facing.

Now, why OpenFire would be terminating your stream when using accents is another matter, which I don't know the answer to. I did find this: http://community.igniterealtime.org/thread/44447

giampaolo commented 12 years ago

I also bumped into this problem by using sleek cset 6f72c05ebf6f464c59f6d647b5dd0c2f8d57744a (1 sep 2011). This is very annoying as I get tons of alarms from my log monitor and the only thing I can do is restart the app.

coffeeowl commented 12 years ago

Same for me, I am writing server component and bumped into this problem right from the start, it is enough to simply stop XMPP server to make component stuck in __read_xml. The problem is that socket returns no data for broken connection as a result ElementTree tries to parse empty input and throws exceptions... not sure why it is not handled in _process, obviously it is an error condition, and _process should exit. I fixed that by throwing SystemExit in FileSocket:

class FileSocket(_fileobject):                                                                                                                                                                         
     def read(self, size=4096):                                                                                                                                                                                                                                                               
        if self._sock is None:                                                                                  
            return None                                                                                         
        data = self._sock.recv(size)                                                                            
       if data:                                                                                                
           return data                                                                                         
        else:                                                                                                                                                         
            raise SystemExit("Broken connection")

I suppose it will not work for Python3, as FileSocket is here to workaround some Python 2.x problems.

legastero commented 12 years ago

coffeeowl: Is that using RC2 or the develop branch?

coffeeowl commented 12 years ago

That was RC2, from PyPI, looks like RC3 is out! I tried it, works fine. At least I can't reproduce this problem with RC3 and ejabberd. So, good news!

giampaolo commented 12 years ago

Sorry but even with RC3 the problem is still there. I get two different exceptions:

[E 2011-11-25 16:19:15,896] m3-gw-xmpp@m3-demo pid=1732 thread:'xmpp-thread' sleekxmpp.basexmpp no element found: line 1, column 18357 Traceback (most recent call last): File "build/bdist.linux-x86_64/egg/sleekxmpp/xmlstream/xmlstream.py", line 1094, in _process if not self.read_xml(): File "build/bdist.linux-x86_64/egg/sleekxmpp/xmlstream/xmlstream.py", line 1127, in read_xml for event, xml in ET.iterparse(self.filesocket, (b'end', b'start')): File "", line 68, in iter SyntaxError: no element found: line 1, column 18357

...and:

[E 2011-11-25 16:19:15,888] m3-gw-xmpp@m3-demo pid=1732 thread:'xmpp-thread' sleekxmpp.xmlstream.xmlstream Error reading from XML stream.

hanwentao commented 12 years ago

I also got this error in RC3. It happens when ClientXMPP.disconnect() is called.

ERROR    Error reading from XML stream.
ERROR    no element found: line 10, column 737
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/sleekxmpp/xmlstream/xmlstream.py", line 1094, in _process
    if not self.__read_xml():
  File "/Library/Python/2.7/site-packages/sleekxmpp/xmlstream/xmlstream.py", line 1127, in __read_xml
    for event, xml in ET.iterparse(self.filesocket, (b'end', b'start')):
  File "<string>", line 86, in next
ParseError: no element found: line 10, column 737

minhnhb commented 12 years ago

Hi, anyone has a workaround for this issue?

alexandr999 commented 12 years ago

I also get this error in Develop branch when ClientXMPP.disconnect () is called.