RealOrangeOne / django-tasks

A reference implementation and backport of background workers and tasks in Django
https://pypi.org/project/django-tasks/
BSD 3-Clause "New" or "Revised" License
209 stars 13 forks source link

Define `priority` semantics #29

Open RealOrangeOne opened 3 weeks ago

RealOrangeOne commented 3 weeks ago

The priority field is currently an opaque positive integer. As currently implemented:

This presents a few issues:

Therefore, the scale needs to be better defined:

jribbens commented 2 weeks ago

I would suggest a default of zero, a range of -128 to +127, and also define some constants, e.g.:

    class Priority(IntEnum):
        LOWEST = -128
        LOWER = -96
        LOW = -48
        DEFAULT = 0
        HIGH = 48
        HIGHER = 96
        HIGHEST = 127

and then people can use the constants to increase the likelihood of priorities set by different packages being suitable when used together, and also they can say things like Priority.HIGH + 1 to mean "a little bit higher priority than HIGH but not as high as HIGHEST".

(One might reasonably say "why would anyone care if a value fits in 8 bits in the modern world" but one might also reasonably reply "why on earth would you need more than 256 different priorities?". There are lessons to be learned I think from things like CSS z-index where people end up using ridiculous values like 999999999 because there are no guidelines whatsoever as to what values to use - there's no way of saying what you actually mean rather than just picking a random number and hoping.)

Also I don't think the DEP defines what the priority actually means. Presumably it means "while tasks of priority X are ready to run, no tasks with priority less than X will be run, no matter how old/overdue they are"? This should probably be made explicit.

RealOrangeOne commented 2 weeks ago

Priority is indeed the biggest measure - not how overdue it is.

I'd not thought of constants - I really like this idea, even if just as a basis. Having 0 in the middle is definitely the way I'm leaning.

256 priorities ought to be enough for anyone (famous last words). Unbounded is going to almost certainly cause issues with other backends. At least with a fixed range it's possible to re-divide that to any smaller range (albeit losing some resolution). Even with the above, you could argue 7 steps is probably enough - fine-grained priorities could end in bike-shedding. I'd be tempted to use 100 either side instead, simply because as humans it's easier to reason about, and the extra 56 priorities probably aren't useful to most.