Redis Sentinel Configuration Error

[amacciola]

Hello,

I am trying to configure EXQ lib to use an external Redis sentinel host.

My exq configurations are:

[
  heartbeat_enable: true,
  node_identifier: Utils.JobQueue.CustomNodeIdentifier,
  scheduler_poll_timeout: 200,
  scheduler_enable: false,
  heartbeat_interval: 60000,
  poll_timeout: 50,
  max_retries: 25,
  shutdown_timeout: 15000,
  start_on_application: false,
  namespace: "exq",
  middleware: [Exq.Middleware.Stats,
   Utils.JobQueue.Middleware.Job,
   Exq.Middleware.Manager, Exq.Middleware.Logger],
  name: Exq,
  mode: :default,
  missed_heartbeats_allowed: 5,
  redis_options: [
    sentinel: [sentinels: ["redis://rfs-workstation:26379"], group: "mymaster"],
    password: "*******"
  ]
]

And i am just setting it up in my application supervision tree:

# this is passed as the child options in the supervison tree
 child_spec_supervisor(
          Exq,
          Exq,
          [exq_configs]
        )

 defp child_spec_supervisor(module_name, id, args \\ []) do
    %{
      id: id,
      start: {
        module_name,
        :start_link,
        args
      },
      restart: :permanent,
      shutdown: 5000,
      type: :supervisor
    }
  end

The error that i am getting on application start up is:

17:26:04.968 [info] Application cogynt_workstation_ingest exited: CogyntWorkstationIngest.Application.start(:normal, []) returned an error: shutdown: failed to start child: Exq
    ** (EXIT) shutdown: failed to start child: Redix
        ** (EXIT) an exception was raised:
            ** (ArgumentError) :host or :port can't be passed as option if :sentinel is used
                (redix) lib/redix/start_options.ex:83: Redix.StartOptions.sanitize_sentinel_opts/2
                (redix) lib/redix/start_options.ex:55: Redix.StartOptions.maybe_sanitize_sentinel_opts/1
                (redix) lib/redix/start_options.ex:38: Redix.StartOptions.sanitize/1
                (redix) lib/redix/connection.ex:25: Redix.Connection.start_link/1
                (stdlib) supervisor.erl:379: :supervisor.do_start_child_i/3
                (stdlib) supervisor.erl:365: :supervisor.do_start_child/2
                (stdlib) supervisor.erl:349: anonymous fn/3 in :supervisor.start_children/2
                (stdlib) supervisor.erl:1157: :supervisor.children_map/4
                (stdlib) supervisor.erl:315: :supervisor.init_children/2
                (stdlib) gen_server.erl:374: :gen_server.init_it/2
                (stdlib) gen_server.erl:342: :gen_server.init_it/6
                (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

yet i am not setting any :host or :port options

[ananthakumaran]

Can you try master branch. The issue was fixed by https://github.com/akira/exq/pull/428

[amacciola]

@ananthakumaran i am using {:exq, "~> 0.14.0"} is the master branch something different ? This seems like the latest release version

[amacciola]

i can switch it to {:exq, git: "git@github.com:akira/exq.git", branch: "master"}, but is everything else in the master branch safe for release ?

[ananthakumaran]

i can switch it to {:exq, git: "git@github.com:akira/exq.git", branch: "master"}, but is everything else in the master branch safe for release ?

Yes, we will probably cut a release in a couple of weeks.

[amacciola]

@ananthakumaran sounds good. Thanks for the help! testing it out now

[amacciola]

@ananthakumaran the logs from my application are still:

EXQ CONFIGS: [
  node_identifier: Utils.JobQueue.CustomNodeIdentifier,
  scheduler_poll_timeout: 200,
  missed_heartbeats_allowed: 5,
  heartbeat_enable: true,
  max_retries: 25,
  shutdown_timeout: 15000,
  heartbeat_interval: 60000,
  poll_timeout: 50,
  middleware: [Exq.Middleware.Stats,
   Utils.JobQueue.Middleware.Job,
   Exq.Middleware.Manager, Exq.Middleware.Logger],
  start_on_application: false,
  scheduler_enable: false,
  name: Exq,
  mode: :default,
  namespace: "exq",
  redis_options: [
    sentinel: [sentinels: ["redis://rfs-workstation:26379"], group: "mymaster"],
    password: "*******"
  ]
]

12:05:47.224 [info] Application ingest exited: Application.start(:normal, []) returned an error: shutdown: failed to start child: Exq
    ** (EXIT) shutdown: failed to start child: Redix
        ** (EXIT) an exception was raised:
            ** (ArgumentError) :host or :port can't be passed as option if :sentinel is used

and i have my mix.exs file as {:exq, git: "git@github.com:akira/exq.git", branch: "master"}

So does this version work with Sentinel or am i just passing the configurations incorrect ?

[ananthakumaran]

@amacciola it does work, we use it in prod. I don't find anything wrong with config posted here. Make sure you are on master and if problem persists, try to trace Exq.Support.Opts.redis_opts/1 function.

[amacciola]

@ananthakumaran i mean i am specifying {:exq, git: "git@github.com:akira/exq.git", branch: "master"} in my mix.exs file and have ran mix.deps.update exq. I do not know any other way i can verify that i am on the master branch.

And for Exq.Support.Opts.redis_opts/1 call. Would that not only be ran after the Exq app is started under the application supervision tree?

[amacciola]

@ananthakumaran okay i found the issue. I am not setting the RedisOptions section in the actual Config.exs I am adding it in dynamically based on if that ENV needs to use Sentinel opts or not. Then i am passing those configs to Exq like this:

 child_spec_supervisor(
          Exq,
          Exq,
          [exq_configs]
        )

defp child_spec_supervisor(module_name, id, args \\ []) do
    %{
      id: id,
      start: {
        module_name,
        :start_link,
        args
      },
      restart: :permanent,
      shutdown: 5000,
      type: :supervisor
    }
  end

But then the Exq is looking directly @ the config.exs section instead of what is being passed i think

Is there anyway i can just make my own RedisConnection and pass the name to Exq and it reuse the connection ?

[amacciola]

@ananthakumaran I figured out a temporary way around this. This is not any issue with this library. It is an issue with our current application using the same config.exs file for both local/dev/prod envs at the moment.

Thank you for the support