ValueError for gateway.list_clusters()

[AurelieAlbert]

Hi Tom, thank you for this example use of dask gateway. I tried to make it run on https://hub.binder.pangeo.io/ and on ocean.pangeo.io, I get the error below when executing the line gateway.list_clusters() in both places. Same thing is happening for cluster = gateway.new_cluster() Is there some environment or configuration issue ?

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-f8cbd85b6152> in <module>
----> 1 gateway.list_clusters()

/srv/conda/envs/pangeo/lib/python3.7/site-packages/dask_gateway/client.py in list_clusters(self, status, **kwargs)
    408         clusters : list of ClusterReport
    409         """
--> 410         return self.sync(self._clusters, status=status, **kwargs)
    411 
    412     def get_cluster(self, cluster_name, **kwargs):

/srv/conda/envs/pangeo/lib/python3.7/site-packages/dask_gateway/client.py in sync(self, func, *args, **kwargs)
    310             )
    311             try:
--> 312                 return future.result()
    313             except BaseException:
    314                 future.cancel()

/srv/conda/envs/pangeo/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
    433                 raise CancelledError()
    434             elif self._state == FINISHED:
--> 435                 return self.__get_result()
    436             else:
    437                 raise TimeoutError()

/srv/conda/envs/pangeo/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
    382     def __get_result(self):
    383         if self._exception:
--> 384             raise self._exception
    385         else:
    386             return self._result

/srv/conda/envs/pangeo/lib/python3.7/site-packages/dask_gateway/client.py in _clusters(self, status)
    387         url = "%s/gateway/api/clusters/%s" % (self.address, query)
    388         req = HTTPRequest(url=url)
--> 389         resp = await self._fetch(req)
    390         return [
    391             ClusterReport._from_json(self._public_address, self.proxy_address, r)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/dask_gateway/client.py in _fetch(self, req)
    360 
    361                     if resp.code in {404, 422}:
--> 362                         raise ValueError(msg)
    363                     elif resp.code == 409:
    364                         raise GatewayClusterError(msg)

ValueError: 404: Not Found

[TomAugspurger]

Thanks for the report. I'll take a look later today.

[TomAugspurger]

OK, for the pangeo binder the issue is that the image from https://github.com/pangeo-gallery/default-binder/blob/master/binder/Dockerfile is using dask-gateway==0.6.1, but the binder has been updated to dask-gateway==0.7.1. We'll need to discuss how we want to handle updates there, and then sync them up. Hopefully today or tomorrow.

I'm not sure what's going on with jupyterhub yet. Seeing strange things with

Exception in callback _HTTPConnection.finish.<locals>.<lambda>(<Future finis...ssl.c:1056)')>) at /srv/conda/envs/notebook/lib/python3.7/site-packages/tornado/simple_httpclient.py:657
handle: <Handle _HTTPConnection.finish.<locals>.<lambda>(<Future finis...ssl.c:1056)')>) at /srv/conda/envs/notebook/lib/python3.7/site-packages/tornado/simple_httpclient.py:657>
Traceback (most recent call last):
  File "/srv/conda/envs/notebook/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/tornado/simple_httpclient.py", line 657, in <lambda>
    fut.add_done_callback(lambda f: final_callback(f.result()))
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/tornado/simple_httpclient.py", line 336, in run
    source_ip=source_ip,
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/tornado/tcpclient.py", line 294, in connect
    False, ssl_options=ssl_options, server_hostname=host
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/tornado/iostream.py", line 1391, in _do_ssl_handshake
    self.socket.do_handshake()
  File "/srv/conda/envs/notebook/lib/python3.7/ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:1056)

openssl recently had a release that caused some issues for projects like distributed. I'm not sure if that's relevant.

[rabernat]

I just hit the same basic issue in https://github.com/pangeo-gallery/physical-oceanography/issues/1.

[rabernat]

If there is any hope of a quick fix for this, I would sincerely appreciate it. The easiest thing I guess would be to update the binder environment?

[TomAugspurger]

Looking into this now.

On Tue, Apr 21, 2020 at 8:42 PM Ryan Abernathey notifications@github.com wrote:

If there is any hope of a quick fix for this, I would sincerely appreciate it. The easiest thing I guess would be to update the binder environment?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/TomAugspurger/pangeo-dask-gateway/issues/2#issuecomment-617493073, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOIS2GO4O2GPTXP6HQILRNZDQPANCNFSM4MNJACZA .

[rabernat]

Cool, thanks! I'm online / on gitter trying to figure out how it all fits together. Happy to chat there if need be.

[rabernat]

Here's what I am now trying:

https://github.com/rabernat/default-binder/blob/update-tag/binder/Dockerfile