After talking with Wu-Jung and Valentina, we think it would be good to explore using a specific Dask Cluster / no Dask Cluster for specific Flows.
One case where this would be useful is if you need to restart a Dask Cluster for a specific Flow (e.g. accumulation of RAM in the ep.open_raw subflow) and not disturb other Flows. Another case where this would be useful is if you have a Flow that needs a Dask Cluster and another Flow that doesn't need one, then you would assign a Dask Cluster to the Flow that needs a Dask Cluster, and not assign a Dask Cluster to the Flow that doesn't need one.
After talking with Wu-Jung and Valentina, we think it would be good to explore using a specific Dask Cluster / no Dask Cluster for specific Flows.
One case where this would be useful is if you need to restart a Dask Cluster for a specific Flow (e.g. accumulation of RAM in the
ep.open_rawsubflow) and not disturb other Flows. Another case where this would be useful is if you have a Flow that needs a Dask Cluster and another Flow that doesn't need one, then you would assign a Dask Cluster to the Flow that needs a Dask Cluster, and not assign a Dask Cluster to the Flow that doesn't need one.