m3dev / gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.
https://gokart.readthedocs.io/en/latest/
MIT License
318 stars 57 forks source link

setting priority random #376

Closed mski-iksm closed 5 months ago

mski-iksm commented 5 months ago

Changing priority randomly to sort the order of task scheduling randomly.

This is useful when jobs are running in parallel. Together with the complete check function during run (complete_check_at_run; default True), it will speeds up parallel execution, because each job will run different task in different order and same tasks will not be ran.

The disadvantage is that the order of execution is scattered, so it changes a little each time when looking at the log.