Closed algal closed 4 days ago
Oh I really like this solution!!
@algal Could you please add some tests into the nb that showcase it keeps thing in order? It can be a super small test.
@ncoop57 I added a test which provides soft confirmation that the generate
function now returns outputs in the order of inputs.
This updates
generate
to return generation results in theinputs
order, rather than in the order generation jobs completed.It still submits the jobs in
inputs
order, so the jobs will start ininputs
order.It relies on
as_completed
to determine, to update the progress bar whenever a job is completed.This does not add any timeout or error handling logic, so we are still relying on the user to build that into job definitions -- e.g., to define jobs such that they will return None after a timeout period or in the event of an error.
I have lightly tested it, by executing 50 jobs with 5 workers, and inspecting results. Let's discuss @ncoop57