search

mapsacosta / htcdaskgateway

A Dask Gateway client extension for heterogeneous cluster mode combining the Kubernetes backend for pain-free scheduler networking, with COFFEA-powered HTCondor workers
2 stars 2 forks source link

Cannot scale if we connect to a cluster from a different kernel #3

Open holzman opened 4 months ago

holzman commented 4 months ago

In one notebook, do:

from htcdaskgateway import HTCGateway
gateway = HTCGateway()
cluster = gateway.new_cluster(image="coffeateam/coffea-base-almalinux8:0.7.22-py3.10")
cluster.scale(2)

And the cluster will scale out appropriately.

However, if you connect from another kernel:

from htcdaskgateway import HTCGateway
gateway = HTCGateway()
clusters = gateway.list_clusters()
cluster = gateway.connect(clusters[0].name)
cluster.scale(3)

It silently fails.