PerlRedis / perl-redis

Perl binding for Redis database
http://metacpan.org/release/Redis/
108 stars 65 forks source link

Error handling is not stack-safe when reconnect is enabled #141

Open allter opened 4 years ago

allter commented 4 years ago

In my setup i saw occurence of the following warnings:

Deep recursion on subroutine "Try::Tiny::try" at /path/to/site_perl/5.18.0/Redis.pm line 275.
Deep recursion on subroutine "Redis::connect" at /path/to/site_perl/5.18.0/Redis.pm line 272.
Deep recursion on subroutine "Redis::__build_sock" at /path/to/site_perl/5.18.0/Redis.pm line 610.
Deep recursion on anonymous subroutine at /path/to/site_perl/5.18.0/Redis.pm line 629.
Deep recursion on subroutine "Redis::__std_cmd" at /path/to/site_perl/5.18.0/Redis.pm line 207.
Deep recursion on subroutine "Redis::__with_reconnect" at /path/to/site_perl/5.18.0/Redis.pm line 250.

As well as weird exception texts containing:

    ...propagated at /path/to/site_perl/5.18.0/Redis.pm line 613.

How to (possibly) reproduce:

I'm not sure whether this error occurs on connection or when trying to use existing connection after haproxy has dropped the backend connection.

v1.995

dams commented 4 years ago

Can you create a test case ? I think it should be possible with using local spawned redis server and without HAproxy ?

allter commented 4 years ago

Currently i don't have much time for it. :(

I think it should be possible to reproduce locally. To do it one can carefully choose Redis error which can be triggerred by configuration of Redis server (because as I investigated, the problem depends on a particular stage of a Redis connection, like authentification). But it's much reliable to make completely mocked Redis implementation.

In our case, the problem was triggered by HAproxy+monitoring in the middle. But even without HAproxy, weird network error can trigger the same.

P.S. It looks like this issue is a duplicate of #97 . If so, the error might be reproduced by just limiting maxclients on a Redis side