HangfireIO / Hangfire

An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required
https://www.hangfire.io
Other
9.43k stars 1.71k forks source link

Hangfire big overhead / delay by scheduling jobs #2405

Open goodstas opened 6 months ago

goodstas commented 6 months ago

Hi, i wrote very simple asp.net application to test the overhead of time of scheduling job with Hangfire. I just used the ASP.NET Web core template and add some to code to do it. I use InMemory storage for jobs if it's matters. The code is below and the results are that there are SECONDS (2-10 seconds) of overhead time. Is this normal? I think it's too big overhead.

`namespace WebHangfire.Controllers { [ApiController] [Route("[controller]")] public class WeatherForecastController : ControllerBase { private static readonly string[] Summaries = new[] { "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching" };

    private readonly ILogger<WeatherForecastController> _logger;

    public WeatherForecastController(ILogger<WeatherForecastController> logger)
    {
        _logger = logger;
    }

    [HttpGet(Name = "GetWeatherForecast")]
    public IEnumerable<WeatherForecast> Get()
    {            
        MyTimer.Start();
        BackgroundJob.Schedule(() => MyTimer.Stop(), TimeSpan.FromSeconds(10));

        return Enumerable.Range(1, 5).Select(index => new WeatherForecast
        {
            Date = DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
            TemperatureC = Random.Shared.Next(-20, 55),
            Summary = Summaries[Random.Shared.Next(Summaries.Length)]
        })
        .ToArray();
    }

}

public static class MyTimer
{
    private static Stopwatch stopwatch = new Stopwatch();

    public static void Start()
    {
        stopwatch.Start();
        //Console.WriteLine($"Current time : {DateTime.Now.ToString("HH:mm:ss.fff")}");
    }

    public static void Stop()
    {
        stopwatch.Stop();

        Console.WriteLine($"{TimeSpan.FromTicks(stopwatch.ElapsedTicks).TotalSeconds}");

        stopwatch.Reset();
    }
}

}`

goodstas commented 6 months ago

I saw that i can play with the polling interval and this significantly improve the delay overhead for scheduled job. What is a reasonable lowest polling interval to put? I implements some statistics server which gets thousands of items per second and needs to calculate the statistics based on these items.

If i put jobs not in the default queue, will it improve / change something?