perrich / Hangfire.MemoryStorage

A memory storage for Hangfire.
Apache License 2.0
131 stars 44 forks source link

Hangfire Dequeueing Order #23

Open jeansimoncollard opened 5 years ago

jeansimoncollard commented 5 years ago

We have more hangfire jobs than our system can process, so they pile up in a queue that never empties completely.

We've noticed that the oldest jobs never get executed. It's only the newly created jobs that do. I started looking in the code and found the code referenced below. If I understand correctly, jobs are processed in descending order (last in first out). Is this normal? I know Hangfire is FIFO by default if we use SQL.

https://github.com/perrich/Hangfire.MemoryStorage/blob/661839c2e83c0e8cd349cd4310fda7ac6b685335/src/Hangfire.MemoryStorage/MemoryStorageConnection.cs#L110

Thank you in advance, Jean-Simon Collard

perrich commented 5 years ago

You've right. The order has been defined like this because others plugins use the same order before my first commit... So, seems to be an old choice.

I'll change the order.

Thanks.

jeansimoncollard commented 5 years ago

Thank you,

I am also unsure if the date represented by "q.AddedAt" is the moment at which the job was enqueued or the moment at which the job was first created. According to the behavior, it looks like it is the time at which the job was created.

DominicRich commented 5 years ago

Any news on when this change might happen? I too found this issue and have been working around it by queuing x high priority jobs and then queuing the rest with the highest priority last. My queue does clear so I don't have the same issue as @jeansimoncollard but this behaviour is frustrating. I've been using this in production without issue (it is just used for a cache warm up so non-essential) so thanks for this.

perrich commented 5 years ago

Not easy as it seems. I've rollbacked my local changes :(. Please contribute if you want a fix.

DominicRich commented 5 years ago

Ah. I assumed it would be a quick fix. I'll try and take a look