aio-libs / aiopg

aiopg is a library for accessing a PostgreSQL database from the asyncio
http://aiopg.readthedocs.io
BSD 2-Clause "Simplified" License
1.39k stars 159 forks source link

Fix ClosableQueue.get on cancellation, close it on Connection.close #859

Closed Pliner closed 3 years ago

Pliner commented 3 years ago

We had the following crutch to force reconnection, it was necessary before the fix #559.

async with connection.cursor() as cursor:
    await cursor.execute(f"LISTEN {self._channel}")
    while True:
        try:
            async with async_timeout.timeout(self._reconnect_timeout):
                message = await connection.notifies.get()
                yield message.payload
        except asyncio.TimeoutError:
            logger.info("Too long delay between notifications. We have to reconnect")
            break

It helped to find the issue in ClosableQueue implementation: ClosableQueue.get doesn't handle cancellation well, get loop.create_task(self._queue.get()) is not cancelled if a cancellation happens in await asyncio.wait([get, self._close_exception], return_when=asyncio.FIRST_COMPLETED).

Task was destroyed but it is pending!
task: <Task pending name='Task-891205' coro=<Queue.get() running at /usr/local/lib/python3.9/asyncio/queues.py:166> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f3a938c2310>()]>>
codecov[bot] commented 3 years ago

Codecov Report

Merging #859 (261c4c7) into master (f68d9d5) will increase coverage by 0.05%. The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #859      +/-   ##
==========================================
+ Coverage   93.36%   93.42%   +0.05%     
==========================================
  Files          11       11              
  Lines        1553     1566      +13     
  Branches      182      184       +2     
==========================================
+ Hits         1450     1463      +13     
  Misses         72       72              
  Partials       31       31              
Impacted Files Coverage Δ
aiopg/connection.py 95.83% <100.00%> (+<0.01%) :arrow_up:
aiopg/utils.py 91.34% <100.00%> (+1.12%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f68d9d5...261c4c7. Read the comment docs.