search

redis / redis-rb

A Ruby client library for Redis
MIT License
3.96k stars 1.03k forks source link

Redis::Cluster attempting Redis connection during asset precompilation #1276

Closed rift137 closed 4 months ago

rift137 commented 4 months ago

During the build process locally and on our CI servers our Redis cluster is not available. Is there a way to prevent redis-clustering from trying to connect to the server during asset precomp and app initialization?

Reminiscent of this

I've tried redis 4.6.0 and redis 5.2.0 with redis-clustering 5.2.0

Ruby 3.0.6, Rails 6.1.7.6

# Using redis 4.6.0 config/initializers/session_store.rb
if Rails.env.production?
  redis_servers = []

  # Split the cluster string and add each node to the array
  Settings.redis_cluster&.split(',')&.each do |node|
    redis_servers << "redis://#{node}"
  end

  # Check if there are any servers configured
  if redis_servers.any?
    # Initialize Redis cluster with the list of servers
    redis_cluster = Redis.new(cluster: redis_servers)

    # Configure the session store to use Redis
    Rails.application.config.session_store :redis_store, servers: redis_cluster, key: '_my_sessions'
  else
    # Log a warning if no Redis servers are configured
    Rails.logger.warn('No Redis servers configured. Session store not initialized.')
  end
end

Settings.redis_cluster ex: "111.0.0.11:7000,111.0.0.11:7001,111.0.0.11:7002,111.0.0.11:7003,111.0.0.11:7004,111.0.0.11:7005"

Docker build stack trace:

 ---> Running in 5642fcb543e9
rake aborted!
Redis::CannotConnectError: Redis client could not connect to any cluster nodes
/application/vendor/bundle/ruby/3.0.0/gems/redis-4.6.0/lib/redis/cluster/slot_loader.rb:22:in `load'
/application/vendor/bundle/ruby/3.0.0/gems/redis-4.6.0/lib/redis/cluster.rb:116:in `fetch_cluster_info!'
/application/vendor/bundle/ruby/3.0.0/gems/redis-4.6.0/lib/redis/cluster.rb:26:in `initialize'
/application/vendor/bundle/ruby/3.0.0/gems/redis-4.6.0/lib/redis.rb:84:in `new'
/application/vendor/bundle/ruby/3.0.0/gems/redis-4.6.0/lib/redis.rb:84:in `initialize'
/application/config/initializers/session_store.rb:9:in `new'
/application/config/initializers/session_store.rb:9:in `<main>'
/application/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:60:in `load'
/application/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:60:in `load'
/application/vendor/bundle/ruby/3.0.0/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:326:in `block in load'
/application/vendor/bundle/ruby/3.0.0/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:299:in `load_dependency'
/application/vendor/bundle/ruby/3.0.0/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:326:in `load'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/engine.rb:681:in `block in load_config_initializer'
/application/vendor/bundle/ruby/3.0.0/gems/activesupport-6.1.7.6/lib/active_support/notifications.rb:205:in `instrument'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/engine.rb:680:in `load_config_initializer'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/engine.rb:634:in `block (2 levels) in <class:Engine>'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/engine.rb:633:in `each'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/engine.rb:633:in `block in <class:Engine>'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/initializable.rb:32:in `instance_exec'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/initializable.rb:32:in `run'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/initializable.rb:61:in `block in run_initializers'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/initializable.rb:50:in `each'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/initializable.rb:50:in `tsort_each_child'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/initializable.rb:60:in `run_initializers'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/application.rb:391:in `initialize!'
/application/config/environment.rb:5:in `<main>'
/application/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
/application/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
/application/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:100:in `register'
/application/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
/application/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
/application/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.6.13/lib/zeitwerk/kernel.rb:34:in `require'
/application/vendor/bundle/ruby/3.0.0/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `block in require'
/application/vendor/bundle/ruby/3.0.0/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:299:in `load_dependency'
/application/vendor/bundle/ruby/3.0.0/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `require'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/application.rb:367:in `require_environment!'
/application/vendor/bundle/ruby/3.0.0/gems/railties-6.1.7.6/lib/rails/application.rb:533:in `block in run_tasks_blocks'
/application/vendor/bundle/ruby/3.0.0/gems/sprockets-rails-3.4.2/lib/sprockets/rails/task.rb:61:in `block (2 levels) in define'
/application/vendor/bundle/ruby/3.0.0/gems/rake-13.1.0/exe/rake:27:in `<top (required)>'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `load'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `kernel_load'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:23:in `run'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/cli.rb:479:in `exec'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/cli.rb:31:in `dispatch'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/cli.rb:25:in `start'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/exe/bundle:49:in `block in <top (required)>'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/application/vendor/bundle/ruby/3.0.0/gems/bundler-2.2.33/exe/bundle:37:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => environment
(See full trace by running task with --trace)

I've tried supplying a dummy cluster to the build locally but haven't been successful with that yet. Thanks

supercaracal commented 4 months ago

Why don't you delay the initialization? Can we do it by passing proc or so? Sorry, I'm not familiar with recent rails.

byroot commented 4 months ago

@supercaracal would be best to delay connecting to the first query.

Otherwise it makes it hard to provide a client to many APIs.

This would also mirror how the standalone client works.

supercaracal commented 4 months ago

I understood. I'll fix it ASAP.

supercaracal commented 4 months ago

The version of 0.10.0 have been released in redis-cluster-client for the fix.

byroot commented 4 months ago

Thank you @supercaracal 🙇