HangfireIO / Hangfire

An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required
https://www.hangfire.io
Other
9.45k stars 1.71k forks source link

TimeoutException: The operation has timed out. #2062

Open krzysiek-b opened 2 years ago

krzysiek-b commented 2 years ago

I've created a sample ASP.NET app and added hangfire

using Hangfire;
using Hangfire.Server;

namespace HangfireServerTestWeb
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var builder = WebApplication.CreateBuilder(args);

            // Add services to the container.
            builder.Services.AddRazorPages();
            builder.Services.AddHangfire(x =>
            {
                x.UseInMemoryStorage();
            });
            builder.Services.AddHangfireServer(x =>
            {
                //x.Queues = new[] { "default", "new" };
            });

            var app = builder.Build();

            // Configure the HTTP request pipeline.
            if (!app.Environment.IsDevelopment())
            {
                app.UseExceptionHandler("/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                app.UseHsts();
            }

            app.UseHttpsRedirection();
            app.UseStaticFiles();

            app.UseRouting();

            app.UseAuthorization();

            app.MapRazorPages();

            app.UseHangfireDashboard();

            var jobId = BackgroundJob.Enqueue(
                () => Console.WriteLine("Fire-and-forget!"));

            RecurringJob.AddOrUpdate(
                () => Console.WriteLine("Recurring!"),
                Cron.Daily);

            app.Run();
        }
    }
}

Then I added few lines of code to remove server like this

                var jobStorage = JobStorage.Current;
                var monitoringApi = jobStorage.GetMonitoringApi();
                foreach (var serverDto in monitoringApi.Servers())
                {
                    jobStorage.GetConnection().RemoveServer(serverDto.Name);
                }

the server gets removed but after a while hangfire dashbard stops responding and ends with exception

An unhandled exception occurred while processing the request.
TimeoutException: The operation has timed out.
Hangfire.InMemory.InMemoryDispatcher.QueryAndWait(Func<InMemoryState, object> query)

Stack Query Cookies Headers Routing
TimeoutException: The operation has timed out.
Hangfire.InMemory.InMemoryDispatcher.QueryAndWait(Func<InMemoryState, object> query)
Hangfire.InMemory.InMemoryDispatcherBase.QueryAndWait<T>(Func<InMemoryState, T> query)
Hangfire.Dashboard.Pages.ServersPage.Execute()
Hangfire.Dashboard.RazorPage.TransformText(string body)
Hangfire.Dashboard.RazorPageDispatcher.Dispatch(DashboardContext context)
Hangfire.Dashboard.AspNetCoreDashboardMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.InvokeCore(HttpContext context, PathString matchedPath, PathString remainingPath)
Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Show raw exception details

Just before exception error is displayed

image
odinserj commented 2 years ago

Thanks for reporting this. Do you see any exceptions logged? May be there was another problem that lead to full stop of query processor.