We have been seeing workers start very long (hour or multi-hour) jobs,
and then have other workers randomly select the same job to start, since
started jobs are fair game as part of our resiliency strategy.
Thus we want workers to prefer unstarted jobs, without imposing an
absolute priority/sort ordering and running the risk of starving out
the started jobs.
This introduces a very basic lottery selection algorithm, where each job
gets some number of tickets proportional to its status (unstarted jobs
get more tickets than started jobs), and the winner is chosen at random,
meaning unstarted jobs are more likely to be chosen. The ticket ratios
for unstarted/started/_ are currently 3:2:1, chosen arbitrarily, and
open to tuning.
We have been seeing workers start very long (hour or multi-hour) jobs, and then have other workers randomly select the same job to start, since started jobs are fair game as part of our resiliency strategy. Thus we want workers to prefer unstarted jobs, without imposing an absolute priority/sort ordering and running the risk of starving out the started jobs.
This introduces a very basic lottery selection algorithm, where each job gets some number of tickets proportional to its status (unstarted jobs get more tickets than started jobs), and the winner is chosen at random, meaning unstarted jobs are more likely to be chosen. The ticket ratios for unstarted/started/_ are currently 3:2:1, chosen arbitrarily, and open to tuning.