6,000 rpoplpush operations per second

[GabeBigBoxVR]

With only 8 nodes with 20 workers each, we're seeing around 6,000 rpoplpush operation per second on our redis instance. This seems excessively high. I was looking to see if I could find some sort of setting regarding polling interval for jobs, but it looks like it may have been deprecated in the latest version.

Any idea what setting I can tweak to reduce this usage?

        private static void OnConfigureHangfireServer(BackgroundJobServerOptions options)
        {
            options.WorkerCount = Program.Config.WorkerCount;
            options.SchedulePollingInterval = TimeSpan.FromSeconds(5);
            options.HeartbeatInterval = TimeSpan.FromSeconds(5);
            options.ServerCheckInterval = TimeSpan.FromSeconds(5);
        }

[shenjielx]

Hi, do you solved it?

[GabeBigBoxVR]

Not really, we just scaled up the nodes to work around it.

[marcoCasamento]

Just to clarify a bit: those options have no effect on the number of rpoplpush you see in redis. The rpoplpush command is called to dequeue a job from the queued list and eneuque it in the fetched list (RedisConnection:174) so to execute it. If in 8 nodes you are altogether executing 8000 job/sec I expect to see 8000 job/sec on you redis instance.

Differently than SQL Server Storage and others, Hangfire Redis AVOIDS polling so to fetch jobs as fast as it can. there's a built-in mechanism (see RedisSubscription) that leverages Redis pub/sub so to be immediately notified when a new job gets enqueued, so to fetch it as soon as possible, should a worker be free to execute it.

As it stands, the behavior is by design. Just scale up your redis instance.