redis / redis-py

Redis Python client
MIT License
12.68k stars 2.53k forks source link

The timeout settings are not working #3291

Open Bor-is-luv opened 5 months ago

Bor-is-luv commented 5 months ago

Hello! I am using Python 3.11.6, redis-py version 4.6.0 and hiredis 2.0.0. My application I run in Docker Python:3.11.6-slim-bookworm. I set up the timeout settings as follows: socket_connect_timeout=0.008 and socket_timeout=0.01. In addition to this, I set the following parameters: socket_keepalive=True, decode_responses=False. After which I began to measure the operation time of MGET. According to the measurement results, I clearly see that the response time sometimes exceeds 0.1 seconds. See screenshot below.

Снимок экрана 2024-06-25 в 11 37 05

My question is why the timeout settings did not work and the MGET operation took 0.1 seconds instead of 0.01 seconds. To solve this problem, I tried: 1) Disabling hiredis. This didn't help. The response time also sometimes exceeds 0.01 seconds. 2) Passing the setting 'socket_keepalive_options': {socket.TCP_USER_TIMEOUT, 3}. This also didn't help. 3) Studying the original client code. I discovered that in the Connection class there is a method called can_read, which has an argument for timeout, which by default is set to 0, meaning that the timeout is absent and the completion of the operation will be awaited infinitely. I used monkey patching and redefined this method to my own, in which case when the timeout is equal to 0, the socket_timeout setting was used. But this also didn't help.

What else can I do? Can this be a bug in the original client code?

harshit98 commented 2 months ago

@Bor-is-luv

Can you share the code reference for 3rd point ? Ideally, I was also looking into the client code and came across this piece of code, but from where timeout is not being passed into this function - I'm unable to figure that out.

def can_read(self, timeout=0):
        """Poll the socket to see if there's data that can be read."""
        sock = self._sock
        if not sock:
            self.connect()

        host_error = self._host_error()

        try:
            return self._parser.can_read(timeout)
        except OSError as e:
            self.disconnect()
            raise ConnectionError(f"Error while reading from {host_error}: {e.args}")
Bor-is-luv commented 2 months ago

@harshit98 in the third point I tried this solution

from redis.connection import Connection

original_function = Connection.can_read

def function(self: Connection, timeout: float = 0) -> bool:
    if not isinstance(timeout, (int, float)) or timeout <= 0:
        if self.socket_timeout is not None:
            timeout = self.socket_timeout
    return original_function(self, timeout)

Connection.can_read = function