search

HangfireIO / Hangfire

An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required
https://www.hangfire.io
Other
9.4k stars 1.7k forks source link

High CPU Usage , High latency #2286

Open Ommkwn2001 opened 1 year ago

Ommkwn2001 commented 1 year ago

We use Hangfire 1.8.5. We have 3 physical servers, separate for sql server, Hangfire dashboard and windows service for hangfire background server.

Windows service server has 12 Processor and we set 36 background workers.

We have 30 recurring jobs each having different interval. Each recurring job enqueues few more jobs.

It works normally when start (or restart) the service when 36 or more jobs in the queue dashboard shows 36 jobs in “processing” and latency for each job is also very low.

But after couple of hours latency keeps increasing and very less number of jobs are in “processing” while it still shows 36 active workers. And cpu usage goes high.

odinserj commented 1 year ago

I'm afraid only profiling (with dotTrace or any other profiler) can show what's going wrong there. Please feel free to post any profiler traces here, and I will help you with the analysis.

Ommkwn2001 commented 1 year ago

Hangfire_Trace

My SQL server is in Productions. So I told to DBA for check it out but this image is a sample proof of my profiler traces

odinserj commented 1 year ago

Can you please archive the trace and send it as an attachment to support [at] hangfire.io?

Ommkwn2001 commented 1 year ago

I just send an email with attached log file having subject "issue #2286".

Ommkwn2001 commented 1 year ago

The problem was of RazorEngine and not of Hangfire. @odinserj helped in fixing the issue. Thanks @odinserj

intetm commented 11 months ago

The problem was of RazorEngine and not of Hangfire. @odinserj helped in fixing the issue. Thanks @odinserj

Could you describe the problem and the solution in order other users could see what could go wrong?