asyncio_redis should wait for server restart

[crusaderky]

Use case

An asyncio_redis connection with auto_reconnect=True is already connected to the Redis server. The Redis server unexpectedly and randomly crashes and is resurrected shortly thereafter at the same host and port by a nanny, e.g. Kubernetes.

Current asyncio_redis behaviour

Application-level requests to the server fail until the connection is reestablished.

Expected behaviour

Application-level requests are silently enqueued and wait until the connection is reestablished. If a connection timeout were to be implemented in create(), the reconnection attempt should use the same timeout.

POC

Tested on Ubuntu Linux x64, asyncio_redis 0.16.0, redis 5.0.3

import asyncio
import subprocess

import asyncio_redis

async def main():
    server = subprocess.Popen(["redis-server"])
    try:
        client = await asyncio_redis.Pool.create()
        await client.ping()
        server.kill()
        server = subprocess.Popen(["redis-server"])
        await asyncio.sleep(1)
        await client.ping()
    finally:
        server.kill()

if __name__ == "__main__":
    asyncio.run(main())

The above runs with no error. After commenting out the sleep(1): asyncio_redis.exceptions.ConnectionLostError: None