dgpetrie
commented
7 months ago I have been able to get a little further in debugging this. Here is a partial stack just before it hangs in await:
/home/dpetrie/dev/py-vcon/py_vcon_server/tests/test_redis_multiprocess.py(44)connect_and_get_things() -> await get_things(db_pool2) /home/dpetrie/dev/py-vcon/py_vcon_server/tests/test_redis_multiprocess.py(28)get_things() -> things = await client.smembers(DB_JUNK) /usr/local/lib/python3.8/dist-packages/redis/asyncio/client.py(601)execute_command() -> conn = self.connection or await pool.get_connection(command_name, **options) /usr/local/lib/python3.8/dist-packages/redis/asyncio/connection.py(1040)get_connection() -> await self.ensure_connection(connection) /usr/local/lib/python3.8/dist-packages/redis/asyncio/connection.py(1062)ensure_connection() -> await connection.connect() /usr/local/lib/python3.8/dist-packages/redis/asyncio/connection.py(243)connect() -> await self.retry.call_with_retry( /usr/local/lib/python3.8/dist-packages/redis/asyncio/retry.py(59)call_with_retry() -> return await do() /usr/local/lib/python3.8/dist-packages/redis/asyncio/connection.py(650)_connect() -> reader, writer = await asyncio.open_connection( /usr/lib/python3.8/asyncio/streams.py(52)openconnection() -> transport, = await loop.create_connection( /usr/lib/python3.8/asyncio/base_events.py(986)create_connection() -> infos = await self._ensure_resolved( /usr/lib/python3.8/asyncio/base_events.py(1365)_ensure_resolved() -> return await loop.getaddrinfo(host, port, family=family, type=type, /usr/lib/python3.8/asyncio/base_events.py(825)getaddrinfo() -> return await self.run_in_executor(
/usr/lib/python3.8/asyncio/base_events.py(782)run_in_executor() -> return futures.wrap_future(
It seems to be hanging waiting for the result of getaddrinfo. To reiterate this is occurring in the child process that was forked. The executor argument to run_in_executor is None. It gets the executor from self._default_executor. I had another, perhaps crazy, hypothesis that the executor used in run_in_executor was one from the parent process instead of from the child process. However, I am not sure how to prove or disproved that.
executor = <concurrent.futures.thread.ThreadPoolExecutor object at 0x7fab2b263040>
I apologize if I am doing something stupid or unsupported with asyncio, multiprocessing or redis.asyncio. If so please, help me understand what I am doing wrong.
Version: redis.asyncio ver. 5.0.1
Platform: Python 3.8.10 on Ubuntu 20.04.6 LTS
Description: The following steps are taken to reproduce this problem: 1) In the parent process: create an async ConnectionPool, create an async redis client, connect to Redis and perform a command. 2) Using concurrent.futures.ProcessPoolExecutor, fork a child process and in that child process: create an async ConnectionPool, create an async redis client, attempting to connect to Redis and perform a command will hang.
My speculation is that there is some contamination of the socket or connection created in the parent process by Python asyncio or redis.asyncio that causes this problem in the child process. However, I do not know either python package well enough to debug this much further.
Included link is a file which can be run from pytest or in the command line that reproduces the problem. I am sorry that I could not reduce the code to anything much smaller. Though this is greatly reduces from the original code. Changing the variable test_redis_in_parent from True to False will make this problem go away. The difference being that the command/connection is not made in the parent process before the child is forked off.
Link provided as python attachments are not allowed: https://raw.githubusercontent.com/py-vcon/py-vcon/acba392bfd2f966db7820e8a9595c8c0ad5df41d/py_vcon_server/tests/test_redis_multiprocess.py