bacalhau-project / bacalhau

Compute over Data framework for public, transparent, and optionally verifiable computation
https://docs.bacalhau.org
Apache License 2.0
642 stars 85 forks source link

Allow a node to run multiple executions for the same job #4102

Open wdbaruni opened 1 week ago

wdbaruni commented 1 week ago

Today we filter out a node if it already has executed the job before, which made sense when we used Count > 1 for job verification, but that is no longer the case.

The goal here is to allow the scheduler to select the same node but with lower rank to spread the work. The first iteration is to select the node after the job is queued and re-evaluated. Meaning we will allocate an execution per node serially and only one execution per scheduling round as it is much simpler to implement.

Future iteration is to allow alllocating multiple executiosn to the same node in the same round