Exception when cogen-chat times out on pull method.

[GoogleCodeExporter]

I'm using the latest trunk version of cogen and the included cogen-chat 
example.  I'm running it 
on Python 2.5 (framework build from python.org) on Mac OS 10.5.6.

When I try out cogen-chat with a single client and allow the pull method to 
time out twice (i.e. 
logging in, typing a message, then waiting 20 seconds), the server throws an 
exception.  Initially, 
I was receiving the same error as described in issue 6:
File 
"/Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py", 
line 
55, in run
    self.handle_error_event(act, 'Hang up.', ConnectionClosed)
NameError: global name 'ConnectionClosed' is not defined

This was an easy fix (adding the missing import to the top of poll_impl.py).  I 
then got the 
following warning, followed by an exception:

/Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py:29:

UserWarning: fd remove error: KeyError(5,)
  warnings.warn("fd remove error: %r" % e)
(most of the stack trace omitted)
File 
"/Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py", 
line 
58, in run
    self.handle_error_event(act, 'Invalid descriptor.')
  File "/Users/cameron/Development/cogen-read-only/cogen/core/proactors/base.py", line 
307, in handle_error_event
    del self.tokens[act]
KeyError: <Recv at 0x1FDA800 with buff:None len:8192 sock:<socket at 0x2073D00> 
last_update:'n/a' coro:<readline Coroutine instance at 0x020762F0 wrapping 
<generator object 
at 0x2073df0>, state: FAILED> flags:'n/a' timeout:None coro:<readline Coroutine 
instance at 
0x020762F0 wrapping <generator object at 0x2073df0>, state: FAILED> 
weak_timeout:True 
delta:'n/a' last_checkpoint:'n/a' prio:3 state:0>

Looking at poll_impl.py again, around line 55, I see the following code:
                if ev & POLLHUP:
                    self.handle_error_event(act, 'Hang up.', ConnectionClosed)
                if ev & POLLNVAL:
                    self.handle_error_event(act, 'Invalid descriptor.')
                elif ev & POLLERR:
                    self.handle_error_event(act, 'Unknown error.')
                else:

It seems that the second 'if' should be an 'elif'.  If I make this change, I 
just get a different 
exception, however.

I successfully ran the cogen tests; however, for a number of them received a 
similar warning as 
above:

test_http11_kalive (tests.test_wsgi.FileWrapperTest_PrioFIRST_PollProactor) ... 
/Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py:29:

UserWarning: fd remove error: KeyError(25,)
  warnings.warn("fd remove error: %r" % e)
ok

Is this a nuance of the Mac/BSD sockets?

Original issue reported on code.google.com by cameronk...@gmail.com on 29 Dec 2008 at 5:00

[GoogleCodeExporter]

I worked around this by installing py-kqueue, which presumably caused cogen to 
now use kqueue instead of 
poll.  This doesn't seem to have the same problem.  Hurray!

Original comment by cameronk...@gmail.com on 29 Dec 2008 at 5:14

[GoogleCodeExporter]

Thanks for the report. 

I'll try to get my hands on a macos to get to the bottom of this (i use a 
freebsd
machine instead at the moment).

Just by looking at the report it would appear that somehow on disconnect 
(onhup) the
fd gets somehow in the poll list even if it was previously removed. And that 
should
never ever happen.

The import and elif fixes are in trunk. So what do you get in the cogen-chat 
app with
those fixes ?

Original comment by ionel...@gmail.com on 29 Dec 2008 at 9:59

[GoogleCodeExporter]

I get the following warning and exception:

Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py:30: 
UserWarning: fd 
remove error: KeyError(5,)
  warnings.warn("fd remove error: %r" % e)
(stack trace snipped)
File "/Users/cameron/Development/cogen-read-only/cogen/core/schedulers.py", 
line 216, in iter_run
    urgent = self.proactor.run(timeout = self.next_timer_delta())
  File "/Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py", line 55, in run
    act = self.shadow.pop(fd)
KeyError: 5

Original comment by cameronk...@gmail.com on 29 Dec 2008 at 3:34

[GoogleCodeExporter]

Do you have the full stacktrace? 

Original comment by ionel...@gmail.com on 29 Dec 2008 at 5:39

[GoogleCodeExporter]

Starting server in PID 2218.
Adding new client: <chatapp.controllers.chat.Client instance at 0x2072b48>
'test'
['guest: test']
/Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py:30:
 UserWarning: fd 
remove error: KeyError(4,)
  warnings.warn("fd remove error: %r" % e)
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/Current/bin/paster", line 8, in <module>
    load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PasteScript-1.7.3-
py2.5.egg/paste/script/command.py", line 84, in run
    invoke(command, command_name, options, args[1:])
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PasteScript-1.7.3-
py2.5.egg/paste/script/command.py", line 123, in invoke
    exit_code = runner.run(args)
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PasteScript-1.7.3-
py2.5.egg/paste/script/command.py", line 218, in run
    result = self.command()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PasteScript-1.7.3-
py2.5.egg/paste/script/serve.py", line 303, in command
    serve()
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PasteScript-1.7.3-
py2.5.egg/paste/script/serve.py", line 287, in serve
    server(app)
  File "/Users/cameron/Development/cogen-read-only/cogen/web/wsgi.py", line 801, in serve
    runner.run()
  File "/Users/cameron/Development/cogen-read-only/cogen/web/wsgi.py", line 772, in run
    self.sched.run()
  File "/Users/cameron/Development/cogen-read-only/cogen/core/schedulers.py", line 234, in run
    for _ in self.iter_run():
  File "/Users/cameron/Development/cogen-read-only/cogen/core/schedulers.py", line 216, in iter_run
    urgent = self.proactor.run(timeout = self.next_timer_delta())
  File "/Users/cameron/Development/cogen-read-only/cogen/core/proactors/poll_impl.py", line 55, in run
    act = self.shadow.pop(fd)
KeyError: 4

Original comment by cameronk...@gmail.com on 29 Dec 2008 at 6:14

[GoogleCodeExporter]

This actually was a bigger problem with error handling (fds were improperly 
removed
leaving garbage behind and some other minor problems) on multiplexers that 
didn't
have ONESHOT flag for watched fds. 

I fixed this in trunk and added a test for this problem. 

Original comment by ionel...@gmail.com on 30 Dec 2008 at 11:57

• Changed state: Fixed