aio-libs / aioelasticsearch

aioelasticsearch-py wrapper for asyncio
https://pypi.python.org/pypi/aioelasticsearch
MIT License
138 stars 41 forks source link

Unknown AIOHttpConnection type errors #100

Open smlbiobot opened 6 years ago

smlbiobot commented 6 years ago

In my app which uses aioelasticsearch, I sometimes get these errors but I can’t figure out if there is something wrong with my app or the lib. Specifically, how do I interpret what is causing it / what it means?

python:   File "/home/sml/virtualenvs/cr-api-web/lib/python3.6/site-packages/aioelasticsearch/transport.py", line 338, in perform_request
python:     ignore=ignore, timeout=timeout, headers=headers,
python:   File "/home/sml/virtualenvs/cr-api-web/lib/python3.6/site-packages/aioelasticsearch/transport.py", line 252, in _perform_request
python:     connection = await self.get_connection()
python:   File "/home/sml/virtualenvs/cr-api-web/lib/python3.6/site-packages/aioelasticsearch/transport.py", line 236, in get_connection
python:     return self.connection_pool.get_connection()
python:   File "/home/sml/virtualenvs/cr-api-web/lib/python3.6/site-packages/aioelasticsearch/pool.py", line 96, in get_connection
python:     self.resurrect()
python:   File "/home/sml/virtualenvs/cr-api-web/lib/python3.6/site-packages/aioelasticsearch/pool.py", line 82, in resurrect
python:     self.dead.put_nowait((timestamp, connection))
python:   File "/usr/lib/python3.6/asyncio/queues.py", line 150, in put_nowait
python:     self._put(item)
python:   File "/usr/lib/python3.6/asyncio/queues.py", line 231, in _put
python:     heappush(self._queue, item)
python: TypeError: '<' not supported between instances of 'AIOHttpConnection' and 'AIOHttpConnection'
hellysmile commented 6 years ago

What exact Python version You are using?

hellysmile commented 6 years ago

Can You provide an example which fails this way?

smlbiobot commented 6 years ago

I am using Python 3.6.2. Example is a bit hard for me to create as it generally happens on some background tasks that I am running. And 99% of those tasks run successfully, but it fails every now and then. The actual code is something very simple:

    await es.index(
        index=es_index_name('player_stats'),
        doc_type="player_stats",
        body=body,
        id=body.get('id'),
    )

where body is just a dict.

My main question is: What does this class do exactly? Does it have to do with available connection pool / timeouts for ES?

If it’s related to timeout, I will try to see if I should add nodes. But I have already seen previously that when it timeouts, it throws the timeout exception. I tried reading through the source but I am not proficient enough to debug this.

hellysmile commented 6 years ago

I found an issue, seems queue.PriorityQueue and asyncio.PriorityQueue works little bit different , which was ported from original driver. Ill try to provide a fix for that this weekend

smlbiobot commented 6 years ago

Sweet—thank you!