search

Namoshek / laravel-redis-sentinel

An extension of Laravels Redis driver which supports connecting to a Redis master through Redis Sentinel.
MIT License
19 stars 16 forks source link

Migrating from monospice/laravel-redis-sentinel-drivers library #18

Closed devnicolas1 closed 1 year ago

devnicolas1 commented 1 year ago

Hello! I'm currently migrating a project from monospice/laravel-redis-sentinel-drivers to this library, but I'm having a really hard time fully understanding how I should adapt all my configs. I would deeply appreciate any sort of help, if possible.

Currently, my database.php file looks like this:

'redis' => [
        'retry_count'    => env('REDIS_RETRY_COUNT', 20),
        'retry_sleep_ms' => env('REDIS_RETRY_SLEEP_MS', 200),
        'client' => env('REDIS_CLIENT', 'phpredis-sentinel'),
        'options' => [
            'cluster' => env('REDIS_CLUSTER', 'redis'),
            'prefix' => env('REDIS_PREFIX', Str::slug(env('APP_NAME'), '_').'_database_'),
        ],

        'default' => [
            'password' => env('REDIS_BROADCAST_PASSWORD', $password),
            'port' => env('REDIS_BROADCAST_PORT', $port),
            'database' => env('REDIS_QUEUE_DATABASE', $database),
            'options' => [
                'prefix' => env('REDIS_PREFIX', Str::slug(env('APP_NAME'), '_').'_default_'),
            ],
            'sentinel_host' => env('REDIS_SENTINEL_HOST', $host),
            'sentinel_port' => (int) env('REDIS_SENTINEL_PORT', $port),
            'sentinel_service' => env('REDIS_SENTINEL_SERVICE', $service),
            'sentinel_timeout' => (float) env('REDIS_SENTINEL_TIMEOUT', 0.1),
            'sentinel_persistent' => env('REDIS_SENTINEL_PERSISTENT'),
            'sentinel_retry_interval' => env('REDIS_SENTINEL_RETRY_INTERVAL', 1000),
            'sentinel_read_timeout' => env('REDIS_SENTINEL_READ_TIMEOUT', 0),
            'sentinel_password' => env('REDIS_SENTINEL_PASSWORD'),
        ],

        'cache' => [
            'host' => env('REDIS_BROADCAST_HOST', $host),
            'password' => env('REDIS_BROADCAST_PASSWORD', $password),
            'port' => env('REDIS_BROADCAST_PORT', $port),
            'database' => env('REDIS_CACHE_DATABASE', $database),
            'sentinel_host' => env('REDIS_SENTINEL_HOST', $host),
            'sentinel_port' => (int) env('REDIS_SENTINEL_PORT', $port),
            'sentinel_service' => env('REDIS_SENTINEL_SERVICE', $service),
            'sentinel_timeout' => (float) env('REDIS_SENTINEL_TIMEOUT', 0.1),
            'sentinel_persistent' => env('REDIS_SENTINEL_PERSISTENT'),
            'sentinel_retry_interval' => env('REDIS_SENTINEL_RETRY_INTERVAL', 1000),
            'sentinel_read_timeout' => env('REDIS_SENTINEL_READ_TIMEOUT', 0),
            'sentinel_password' => env('REDIS_SENTINEL_PASSWORD'),
        ],

There also other entries (like session, broadcasting), but they are essentially the same as cache.

In my cache.php file, I have:

        'redis' => [
            'driver' => 'redis',
            'connection' => 'cache',
        ],

        'phpredis-sentinel' => [
            'driver' => 'redis',
            'connection' => env(
                'CACHE_REDIS_SENTINEL_CONNECTION',
                env('CACHE_REDIS_CONNECTION', 'cache')
            ),
        ],

The relevant part of my .env is:

REDIS_CLIENT=phpredis-sentinel
REDIS_CLUSTER=predis
REDIS_DRIVER=phpredis-sentinel
CACHE_DRIVER=phpredis-sentinel

Since the project is already running, I know things like the host, password, database and etc are correct. What is really confusing me, is how exactly I should add the new driver (phpredis-sentinel) into my configs. For instance, when we were still using the Monospice library, the cache.php file had:

        'redis-sentinel' => [
            'driver' => 'redis-sentinel
            'connection' => env(
                'CACHE_REDIS_SENTINEL_CONNECTION',
                env('CACHE_REDIS_CONNECTION', 'cache')
            ),
        ],

The .env looked like:

REDIS_CLIENT=predis
REDIS_CLUSTER=predis
REDIS_DRIVER=redis-sentinel
CACHE_DRIVER=redis-sentinel

When switching libraries, I first thought that I'd have to switch all my references of redis-sentinel as a driver to phpredis-sentinel, while also defining the client and the other sentinel_* variables mentioned in the README. Thing is, in my cache.php, when I try to use 'driver' => 'phpredis-sentinel, I receive the error Driver [phpredis-sentinel] is not supported. The only driver I managed to make work, is redis.

So, my main question here is: is defining the client in database.php enough, or should I actually be able to manually define the drivers in cache.php and the other files like that (session.php, etc)? If the driver should be defined (and not be simply redis), can you spot any possible reason for why Laravel is returning a not supported message? Library is installed, of course.

Since the README doesn't mention any special configuration for files like cache.php or the .env, I really don't know exactly how should I "simplify" all the past configs to adapt it to the new library. Should I really just stick with redis as a driver for everything and the phpredis-sentinel client defined in database.php will do all the magic?

I will gladly send any other details that might be needed, and even add some more content to the README if you judge it might be interesting for people who might be in a spot like mine.

Regardless of all that, thanks for the awesome library! :)

Namoshek commented 1 year ago

What Laravel version are you running on?

devnicolas1 commented 1 year ago

Currently trying on Laravel 9 since its the stable and working version of the project, but the plan is to put it into Laravel 10 right after making it work. Reason for this is that, since the project is already stable on Laravel 9, we thought it would be easier to make it work first, and then upgrading to 10.

Namoshek commented 1 year ago

Ok, I see - your upgrade strategy should be no problem at all.

Regarding the configuration: There are five config files that normally - in one way or another - define or reference the Redis configuration:

The first one, config/database.php, configures the actual Redis connections, while the other four only reference the connections of the former. So CACHE_DRIVER=phpredis-sentinel is wrong and should simply be redis (same for all other occurrences). The only reference to phpredis-sentinel should be the client in config/database.php.

What I also find odd about your configuration is the REDIS_CLUSTER=predis part. The package is entirely incompatible with predis (since predis supports Redis Sentinel out of the box already, as far as I know). Maybe remove this option all-together...

devnicolas1 commented 1 year ago

So, doing said changes, I'm having the following error:

 RedisException 
Operation timed out
at vendor/laravel/framework/src/Illuminate/Redis/Connectors/PhpRedisConnector.php:161
  157▕         if (version_compare(phpversion('redis'), '5.3.0', '>=') && ! is_null($context = Arr::get($config, 'context'))) {
  158▕             $parameters[] = $context;
  159▕         }
  160▕ 
➜ 161▕         $client->{$persistent ? 'pconnect' : 'connect'}(...$parameters);
  162▕     }
  163▕ 
  164▕     /**
  165▕      * Create a new redis cluster instance.
    +42 vendor frames 
43  artisan:37
    Illuminate\Foundation\Console\Kernel::handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

The logs do show that the request went through the library, which I suppose means that the client is being correctly used.

[2023-08-25 17:51:07] local.ERROR: Operation timed out {"exception":"[object] (RedisException(code: 0): Operation timed out at /srv/vendor/laravel/framework/src/Illuminate/Redis/Connectors/PhpRedisConnector.php:161)
[stacktrace]
#0 /srv/vendor/laravel/framework/src/Illuminate/Redis/Connectors/PhpRedisConnector.php(161): Redis->connect('17.123.0.54', '6379', 0.0, NULL, 0, 0.0)
#1 /srv/vendor/laravel/framework/src/Illuminate/Redis/Connectors/PhpRedisConnector.php(90): Illuminate\\Redis\\Connectors\\PhpRedisConnector->establishConnection(Object(Redis), Array)
#2 /srv/vendor/laravel/framework/src/Illuminate/Support/helpers.php(306): Illuminate\\Redis\\Connectors\\PhpRedisConnector->Illuminate\\Redis\\Connectors\\{closure}(Object(Redis))
#3 /srv/vendor/laravel/framework/src/Illuminate/Redis/Connectors/PhpRedisConnector.php(131): tap(Object(Redis), Object(Closure))
#4 /srv/vendor/namoshek/laravel-redis-sentinel/src/Connectors/PhpRedisSentinelConnector.php(71): Illuminate\\Redis\\Connectors\\PhpRedisConnector->createClient(Array)
#5 /srv/vendor/namoshek/laravel-redis-sentinel/src/Connectors/PhpRedisSentinelConnector.php(32): Namoshek\\Redis\\Sentinel\\Connectors\\PhpRedisSentinelConnector->createClient(Array)
#6 /srv/vendor/namoshek/laravel-redis-sentinel/src/Connectors/PhpRedisSentinelConnector.php(36): Namoshek\\Redis\\Sentinel\\Connectors\\PhpRedisSentinelConnector->Namoshek\\Redis\\Sentinel\\Connectors\\{closure}()
#7 /srv/vendor/laravel/framework/src/Illuminate/Redis/RedisManager.php(112): Namoshek\\Redis\\Sentinel\\Connectors\\PhpRedisSentinelConnector->connect(Array, Array)
#8 /srv/vendor/laravel/framework/src/Illuminate/Redis/RedisManager.php(91): Illuminate\\Redis\\RedisManager->resolve('cache')
#9 /srv/vendor/laravel/framework/src/Illuminate/Cache/RedisStore.php(262): Illuminate\\Redis\\RedisManager->connection('cache')
#10 /srv/vendor/laravel/framework/src/Illuminate/Cache/RedisStore.php(237): Illuminate\\Cache\\RedisStore->connection()
#11 /srv/vendor/laravel/framework/src/Illuminate/Cache/Repository.php(680): Illuminate\\Cache\\RedisStore->flush()
#12 /srv/vendor/laravel/framework/src/Illuminate/Cache/Console/ClearCommand.php(80): Illuminate\\Cache\\Repository->__call('flush', Array)
#13 /srv/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\\Cache\\Console\\ClearCommand->handle()
#14 /srv/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#15 /srv/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))
#16 /srv/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))
#17 /srv/vendor/laravel/framework/src/Illuminate/Container/Container.php(661): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)
#18 /srv/vendor/laravel/framework/src/Illuminate/Console/Command.php(183): Illuminate\\Container\\Container->call(Array)
#19 /srv/vendor/symfony/console/Command/Command.php(326): Illuminate\\Console\\Command->execute(Object(Symfony\\Component\\Console\\Input\\ArrayInput), Object(Illuminate\\Console\\OutputStyle))
#20 /srv/vendor/laravel/framework/src/Illuminate/Console/Command.php(153): Symfony\\Component\\Console\\Command\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArrayInput), Object(Illuminate\\Console\\OutputStyle))
#21 /srv/vendor/laravel/framework/src/Illuminate/Console/Concerns/CallsCommands.php(68): Illuminate\\Console\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArrayInput), Object(Symfony\\Component\\Console\\Output\\NullOutput))
#22 /srv/vendor/laravel/framework/src/Illuminate/Console/Concerns/CallsCommands.php(40): Illuminate\\Console\\Command->runCommand('cache:clear', Array, Object(Symfony\\Component\\Console\\Output\\NullOutput))
#23 /srv/vendor/laravel/framework/src/Illuminate/Foundation/Console/OptimizeClearCommand.php(48): Illuminate\\Console\\Command->callSilent('cache:clear')
#24 /srv/vendor/laravel/framework/src/Illuminate/Console/View/Components/Task.php(37): Illuminate\\Foundation\\Console\\OptimizeClearCommand->Illuminate\\Foundation\\Console\\{closure}()
#25 /srv/vendor/laravel/framework/src/Illuminate/Console/View/Components/Factory.php(58): Illuminate\\Console\\View\\Components\\Task->render('cache', Object(Closure))
#26 /srv/vendor/laravel/framework/src/Illuminate/Foundation/Console/OptimizeClearCommand.php(52): Illuminate\\Console\\View\\Components\\Factory->__call('task', Array)
#27 /srv/vendor/laravel/framework/src/Illuminate/Collections/Traits/EnumeratesValues.php(235): Illuminate\\Foundation\\Console\\OptimizeClearCommand->Illuminate\\Foundation\\Console\\{closure}(Object(Closure), 'cache')
#28 /srv/vendor/laravel/framework/src/Illuminate/Foundation/Console/OptimizeClearCommand.php(52): Illuminate\\Support\\Collection->each(Object(Closure))
#29 /srv/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\\Foundation\\Console\\OptimizeClearCommand->handle()
#30 /srv/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#31 /srv/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure(Object(Closure))
#32 /srv/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Illuminate\\Foundation\\Application), Array, Object(Closure))
#33 /srv/vendor/laravel/framework/src/Illuminate/Container/Container.php(661): Illuminate\\Container\\BoundMethod::call(Object(Illuminate\\Foundation\\Application), Array, Array, NULL)
#34 /srv/vendor/laravel/framework/src/Illuminate/Console/Command.php(183): Illuminate\\Container\\Container->call(Array)
#35 /srv/vendor/symfony/console/Command/Command.php(326): Illuminate\\Console\\Command->execute(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))
#36 /srv/vendor/laravel/framework/src/Illuminate/Console/Command.php(153): Symfony\\Component\\Console\\Command\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Illuminate\\Console\\OutputStyle))
#37 /srv/vendor/symfony/console/Application.php(1063): Illuminate\\Console\\Command->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#38 /srv/vendor/symfony/console/Application.php(320): Symfony\\Component\\Console\\Application->doRunCommand(Object(Illuminate\\Foundation\\Console\\OptimizeClearCommand), Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#39 /srv/vendor/symfony/console/Application.php(174): Symfony\\Component\\Console\\Application->doRun(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#40 /srv/vendor/laravel/framework/src/Illuminate/Console/Application.php(102): Symfony\\Component\\Console\\Application->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#41 /srv/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(155): Illuminate\\Console\\Application->run(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#42 /srv/artisan(37): Illuminate\\Foundation\\Console\\Kernel->handle(Object(Symfony\\Component\\Console\\Input\\ArgvInput), Object(Symfony\\Component\\Console\\Output\\ConsoleOutput))
#43 {main}
"}

What I'm doing as a test is simply:

> use Illuminate\Support\Facades\Redis;
> $redis = Redis::connection();

This would usually return a valid connection while using the Monospice library

I also made a few tests with REDIS_CLUSTER variable, and it didn't seem to have any effect so far. As I'm not totally sure of the effect of it (since I'm not the one who originally configured it all), I'll keep testing to see what exactly its doing

I'm not sure if this is something exactly related to the library at this point, so although I would totally appreciate any help possible, I totally understand if you don't want to/don't have the time to this. Thanks for the help anyway!

Namoshek commented 1 year ago

The operation timed out error sounds like a firewall issue to me. Are you sure that you are able to connect from the machine you are testing this on? Maybe run telnet <ip> <port> for both the Redis Sentinel and Redis ports and all IPs.

Namoshek commented 1 year ago

Maybe as a little more help, here the configuration we've been using successfully.

config/database.php:

return [

    'redis' => [

        'client' => 'phpredis-sentinel',

        'default' => [
            'sentinel_host' => env('REDIS_SENTINEL_HOST', '127.0.0.1'),
            'sentinel_port' => intval(env('REDIS_SENTINEL_PORT', 26379)),
            'sentinel_password' => env('REDIS_SENTINEL_PASSWORD'),
            'sentinel_service' => env('REDIS_SENTINEL_SERVICE', 'mymaster'),
            'sentinel_timeout' => env('REDIS_SENTINEL_TIMEOUT', 0),
            'sentinel_persistent' => env('REDIS_SENTINEL_PERSISTENT'),
            'sentinel_retry_interval' => env('REDIS_SENTINEL_RETRY_INTERVAL', 0),
            'sentinel_read_timeout' => env('REDIS_SENTINEL_READ_TIMEOUT', 0),
            'password' => env('REDIS_PASSWORD'),
            'database' => 0,
        ],
    ]
];
devnicolas1 commented 1 year ago

To conclude all of this: although I couldn't make this library work, I do managed to find a workaround in our project that solved our problems pretty decently. Regardless of that, thank you for all the help! Really appreciate it :)