dask / distributed

A distributed task scheduler for Dask
https://distributed.dask.org
BSD 3-Clause "New" or "Revised" License
1.58k stars 718 forks source link

Cluster.name (and other _cluster_info items) get carried over to subsequent Clusters #6485

Closed graingert closed 2 years ago

graingert commented 2 years ago

( if Cluster.__init__ is not called before Cluster.name.__set__ )

import sys

from distributed.deploy import Cluster

class ExampleCluster(Cluster):
    def __init__(self, name, *args, **kwargs):
        self.name = f"example-cluster-{name}"
        super().__init__(*args, **kwargs)

def main():

    print(
        Cluster()
    )  # Cluster(e174b1b2, '<Not Connected>', workers=0, threads=0, memory=0 B)
    print(
        ExampleCluster("demo")
    )  # ExampleCluster(example-cluster-demo, '<Not Connected>', workers=0, threads=0, memory=0 B)
    print(
        Cluster()
    )  # Cluster(example-cluster-demo, '<Not Connected>', workers=0, threads=0, memory=0 B)
    return 0

if __name__ == "__main__":
    sys.exit(main())

see also https://github.com/dask/dask-kubernetes/issues/516 see also https://github.com/dask/distributed/pull/5305#discussion_r886822632

graingert commented 2 years ago

@jacobtomlinson

mrocklin commented 2 years ago

I'm curious, what was the context here? Where did this come up?