The worker_options param of SSHCluster includes a property "n_workers", however, when set, this would start the same number of workers on each host. This assumes that all the nodes within the cluster share the same resources, which is not always the case.
Need to find a way to use a custom configuration file (can be built myself) that instructs Dask to start a different number of workers on each node.
The worker_options param of SSHCluster includes a property "n_workers", however, when set, this would start the same number of workers on each host. This assumes that all the nodes within the cluster share the same resources, which is not always the case.
Need to find a way to use a custom configuration file (can be built myself) that instructs Dask to start a different number of workers on each node.