As discussed with @debsankha, we agreed that input generators should be iterated at run-time on the host. That way, all input is calculated and incorporated into the dask task graph, to be submitted to the scheduler.
If input generation is costly, it is at the discretion of the user to set up a seperate "experiment" for that.
As discussed with @debsankha, we agreed that input generators should be iterated at run-time on the host. That way, all input is calculated and incorporated into the dask task graph, to be submitted to the scheduler.
If input generation is costly, it is at the discretion of the user to set up a seperate "experiment" for that.