Detach task scheduling from waking up threads

kvark / choir

Task Orchestration Framework

MIT License

52 stars 3 forks source link

Closed kvark closed 2 years ago

kvark commented 2 years ago

The benchmark of running N individual tasks on 2 workers is currently entering this weird state where:

both workers are mostly parked/sleeping
schedule() take a long time since it repeatedly tries to wake up one of them until it grabs the task
this makes the main thread unable to saturate the queue