RhodiumGroup / rhg_compute_tools

Tools for using compute.rhg.com and compute.impactlab.org
MIT License
1 stars 4 forks source link

Allow labels and tolerations in get_cluster #58

Closed delgadom closed 4 years ago

delgadom commented 4 years ago

Usage

To create cluster which will spin up workers with custom labels and tolerations:

client, cluster = rhgk.get_cluster(
    extra_worker_labels={
        'project': 'my-project',
        'phase': 'phase2',
        'job': 'my-big-distributed-job',
    },
    extra_pod_tolerations=[
            {
                "effect": "NoSchedule",
                "key": "k8s.dask.org_dedicated",
                "operator": "Equal",
                "value": "worker-highcpu",
            },
            {
                "effect": "NoSchedule",
                "key": "k8s.dask.org/dedicated",
                "operator": "Equal",
                "value": "worker-highcpu",
            },
    ],
)