IMonitoringApi is returning results in reversing order

[simaoribeiro]

Hi there.

My team is currently implementing metrics related to Hangfire so we can configure alarmistic based on certain types of events. Currently, we're fetching the failed jobs so we can fire alarms when a job from X type fails. Eg.:

scheduler_job_count{type="hangfire",state="failed",job="IProcessNumber.Process"} 181

Our Hangfire implementation uses MongoDB (outside of this repository, I know), however, I believe that there is a problem in the MongoDB implementation of the IMonitoringApi that originated here.

The IMonitoringApi defines, among others, the following methods:

JobList<ProcessingJobDto> ProcessingJobs(int from, int count);
JobList<ScheduledJobDto> ScheduledJobs(int from, int count);
JobList<SucceededJobDto> SucceededJobs(int from, int count);
JobList<FailedJobDto> FailedJobs(int from, int count);
JobList<DeletedJobDto> DeletedJobs(int from, int count);

When looking at these methods, one can think that the results are paginated from the oldest job to the new ones. As an example, if I have 10000 jobs that failed and 5 new jobs failed after the last time I accessed the information I would think that to access the new jobs, I would call the FailedJobs with the from = 10000 and the count = 5.

However, looking at the Hangfire.MongoDB implementation (I know, still not about this repo, we will get there) the query that is made to the database is inversing the order of the array before fetching the results, which means that the method invocation I exemplified above will give me not the most recent 5 failed jobs but the 5 oldest.

https://github.com/Hangfire-Mongo/Hangfire.Mongo/blob/master/src/Hangfire.Mongo/MongoMonitoringApi.cs#L563

I thought that it was an error in the Mongo implementation but then we went to check other databases implementations and realized that this is the behavior in some of them.

MemoryStorage https://github.com/perrich/Hangfire.MemoryStorage/blob/master/src/Hangfire.MemoryStorage/MemoryStorageMonitoringApi.cs#L367

SQLServer https://github.com/HangfireIO/Hangfire/blob/master/src/Hangfire.SqlServer/SqlServerMonitoringApi.cs#L176

So, the question is: Is this the desired behavior for the IMonitoringApi implementations? Or is it an error that was made in the original implementation that was replicated in the other databases like MongoDB?

Honestly, for me, it looks a bit strange that to get the last X jobs from a certain state I need to call the methods with the from argument as 0 all the time and the count with the X value.

Cheers

[simaoribeiro]

I analyzed the implementation of the IMonitoringApi in multiple databases and there are some inconsistencies between them as you can see in the table below. These differences in behavior increase the difficulty of developing code to export metrics that are agnostic to the database being used which is our use case.

Job State	Hangfire.SqlServer	Hangfire.InMemory	Hangfire.MongoDB	Hangfire.MemoryStorage
Deleted	Reverted	Reverted	Reverted	Reverted
Enqueued
Failed	Reverted	Reverted (0.7+)	Reverted	Reverted
Processing			Reverted	Reverted
Scheduled			Reverted	Reverted
Succeeded	Reverted	Reverted	Reverted	Reverted

[odinserj]

​You are right, and I'm afraid there can be other minor differences as well. The thing is that abstractions weren't properly documented yet due to the complexity of the topic, especially consistency / transaction isolation level issues. And the best thing I can advise is to treat Hangfire.SqlServer and Hangfire.Pro.Redis implementations as the reference ones. ​ Reversed order relates to Succeeded / Deleted / Failed jobs only to make it more comfortable to see what's changed in those lists. Because otherwise their first pages will be almost the same most of the time until first entries are expired.

[burningice2866]

@simaoribeiro You can add Hangfire.CompositeC1 to the list and note that all jobs returned from IMonitoringApi are sorted by CreatedAt in descending order.

https://github.com/burningice2866/Hangfire.CompositeC1/blob/master/CompositeC1MonitoringApi.cs#L418