KeyNotFoundException on ScheduledJobs list fetch

[gienec]

Hi, I get the exception KeyNotFoundException when I'm trying to fetch scheduled jobs via monitoring API method ScheduledJobs. Stacktrace:

  at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Hangfire.Redis.RedisMonitoringApi.<>c__DisplayClass9_1.<ScheduledJobs>b__3(SortedSetEntry job)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Hangfire.Redis.RedisMonitoringApi.<>c__DisplayClass9_0.<ScheduledJobs>b__0(IDatabase redis)
   at Hangfire.Redis.RedisMonitoringApi.UseConnection[T](Func`2 action)
   at Hangfire.Redis.RedisMonitoringApi.ScheduledJobs(Int32 from, Int32 count)

Any ideas how it could be fixed plz?

[marcoCasamento]

What version are you using? There is a mess in the source code in branch master after yesterday merge, sorry, hope to fix It very soon

[gienec]

I'm using 1.6.7.24 nuget version.

[gienec]

Also there should be mentioned that this exception is not thrown all the time. It appears then job processing activity is high.

[marcoCasamento]

Looking at the call stack I'd say it happens on RedisMonitoringApi.JobDetails rather than in ScheduledJobs. Can you try to increase the redisSyncTimeOut to 5000 ? i.e. localhost:6379,syncTimeout=5000,allowAdmin=true

Also, what's your setup ? Are you running Redis on the same windows machine as hangfire ?

[gienec]

Redis and Hangfire run on different machines. Also Redis runs on Linux, syncTimeout is default - 1000. I've tried to skipJobDetails and did calls only of SheduledJobs and got exception of KeyNotFoundException.

[gienec]

Increase of syncTimeout didn't help.

[marcoCasamento]

Uhm... maybe the ContinueWith calls aren't somehow awaited correctly, could you verify it inserting a couple of lines in the code ?

        Task.WaitAll(tasks);
//code insert starts here
                if (scheduledJobs.Any(x=> !jobs.ContainsKey(x.Element)))
                {
                    throw new HangFireRedisException("not all job were fetched correctly");
                }
//code insert ends here

                return new JobList<ScheduledJobDto>(scheduledJobs

inside the ScheduledJobs method, and let see if the exception is ever launched

[gienec]

Ok. I'll try to do it later and will report about results.

[marcoCasamento]

Any news on this ?

[gienec]

I stopped to use scheduled jobs. It's hard to reproduce this bug so I'll close it until I'll find out how exactly to reproduce it.