In async mode, connection is not marked as closed when server dies

[haizaar]

I'm writing server failure detection code in momoko and hitting the following issue with psycopg2cffi. It works with the regular psycopg2.

The scenario:

• Create asynchronous connection to postgres server
• Kill postgress server (I mean kill, plain service postgres restart won't do).
• Obtain cursor from connection and execute "SELECT 1".
• After hitting exception - bug: conn.closed should be > 0, white it's equal 0

psycopg2cffi 2.7.2, Python 2.7, cffi 1.2.1, Postgres 9.3 (same behaviour with 9.2 and 9.4).

Reproduction:

from __future__ import print_function
from psycopg2cffi import compat
compat.register()
import psycopg2
import select
import traceback

def wait(conn):
    while 1:
        state = conn.poll()
        if state == psycopg2.extensions.POLL_OK:
            break
        elif state == psycopg2.extensions.POLL_WRITE:
            select.select([], [conn.fileno()], [])
        elif state == psycopg2.extensions.POLL_READ:
            select.select([conn.fileno()], [], [])
        else:
            raise psycopg2.OperationalError("poll() returned %s" % state)

try:
    aconn = psycopg2.connect(dsn="user=foo password=bar host=127.0.0.1 port=5432", async=1)
    wait(aconn)

    print("Kill postgres (sudo pkill -9 -f postgres) and hit Enter...", end="")
    raw_input()

    acur = aconn.cursor()
    acur.execute("SELECT 1")
    wait(aconn)
except Exception as err:
    print("Execution failed: %s" % err)
    traceback.print_exc()
    print("aconn.closed = %s" % aconn.closed)
else:
    print(acur.fetchone())

And let's run it:

$ python asynclose.py 
Kill postgres (sudo pkill -9 -f postgres) and hit Enter...^Z
[1]+  Stopped                 python asynclose.py
$ sudo pkill -9 -f postgres
Killed
$ fg
python asynclose.py

Execution failed: SSL SYSCALL error: EOF detected

Traceback (most recent call last):
  File "asynclose.py", line 29, in <module>
    wait(aconn)
  File "asynclose.py", line 10, in wait
    state = conn.poll()
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 480, in poll
    res = self._poll_query()
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 534, in _poll_query
    ret = self._poll_advance_read(self._is_busy())
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 774, in _is_busy
    ffi.string(libpq.PQerrorMessage(self._pgconn)))
OperationalError: SSL SYSCALL error: EOF detected

aconn.closed = 0
$

Note the aconn.closed = 0.

The similar code but in synchronous mode works fine:

from __future__ import print_function
from psycopg2cffi import compat
compat.register()
import psycopg2
import traceback

try:
    conn = psycopg2.connect(dsn="user=foo password=bar host=127.0.0.1 port=5432")

    print("Kill postgres (sudo pkill -9 -f postgres) and hit Enter...", end="")
    raw_input()

    cur = conn.cursor()
    cur.execute("SELECT 1")
except Exception as err:
    print("Execution failed: %s" % err)
    traceback.print_exc()
    print("conn.closed = %s" % conn.closed)
else:
    print(acur.fetchone())

And running it:

$PS1='$ '
$ 
$ python synclose.py 
Kill postgres (sudo pkill -9 -f postgres) and hit Enter...^Z
[1]+  Stopped                 python synclose.py
$ sudo pkill -9 -f postgres
Killed
$ fg
python synclose.py

Execution failed: SSL SYSCALL error: EOF detected

Traceback (most recent call last):
  File "synclose.py", line 15, in <module>
    cur.execute("SELECT 1")
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.py", line 30, in check_closed_
    return func(self, *args, **kwargs)
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.py", line 252, in execute
    conn._begin_transaction()
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 643, in _begin_transaction
    self._execute_command('BEGIN')
  File "/home/.../venvs/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 660, in _execute_command
    raise exc
DatabaseError: SSL SYSCALL error: EOF detected

conn.closed = 2
$ 

conn.closed is greater than 0 as expected.

[lopuhin]

Thank you for reporting with a way to reproduce! I am traveling right now, will get back to the issue on the next week.

[haizaar]

Do you have any ETA on this bug, please?

[haizaar]

Ping :)

[lopuhin]

No ETA, sorry - I'm not sure I'll be able to look into it any time soon.