search

det-lab / jupyterhub-deploy-kubernetes-jetstream

CDMS JupyterHub deployment on XSEDE Jetstream
0 stars 1 forks source link

Unable to access jupyter lab website #82

Closed zkromerUCD closed 1 year ago

zkromerUCD commented 1 year ago

When trying to access jetstream via http://supercdms.jetstream-cloud.org/, the page loads for a long time, and then the error "supercdms.jetstream-cloud.org took too long to respond." appears in the browser. The browser that I am using is google chrome - Amy also was unable access the webpage and received the same error.

zonca commented 1 year ago

ok, looking into that, not sure yet what is going on, I see the Virtual Machines running but I cannot access them.

zonca commented 1 year ago

I rebooted the instances and now I can access them, but all pods are throwing errors

zonca commented 1 year ago

I have volumes stuck in "detaching" state:

o volume list | grep -v available | grep -v in-use
+--------------------------------------+------------------------------------------+-----------+------+---------------------------------------------------+
| ID                                   | Name                                     | Status    | Size | Attached to                                       |
+--------------------------------------+------------------------------------------+-----------+------+---------------------------------------------------+
| a6443fdb-13da-4141-a1ea-9c2874c62df7 | pvc-a34caa66-fdd2-40c1-9839-ca573474bb7a | detaching |  200 | Attached to kubejetstream-k8s-node-1 on /dev/sdf  |
| 0b7ee629-520e-4a4b-bee1-9aae938406f7 | pvc-4344d4f1-fdeb-44df-95ba-71f791b89bdb | detaching |   10 | Attached to kubejetstream-k8s-node-1 on /dev/sdj  |
| 3bfecc11-7ce4-4b14-9873-8151fd92c8ca | pvc-f8c3e334-8bee-4e0c-bc9e-48d44c4e2624 | detaching |   10 | Attached to kubejetstream-k8s-node-1 on /dev/sdk  |
| 9d1d8989-45f8-4127-b699-43d416967032 | pvc-a98a8f7a-9b89-4d27-9a9c-5557a501c314 | detaching |   10 | Attached to kubejetstream-k8s-node-1 on /dev/sdg  |
| 5079f23f-6758-412b-b9f0-316e55400558 | pvc-59c503f8-2793-4b69-bd86-927f8c0fd8c4 | detaching |   10 | Attached to kubejetstream-k8s-node-1 on /dev/sdh  |
| 886261f2-fb84-4ce5-93bb-a4d32fbda76b | pvc-6e9eb56a-82dc-4960-8bfe-8ee66db83521 | detaching |   10 | Attached to kubejetstream-k8s-node-1 on /dev/sdi  |
+--------------------------------------+------------------------------------------+-----------+------+---------------------------------------------------+
zonca commented 1 year ago

I asked for support, I cannot fix it myself, I'll keep this updated. JS2 is in early operations, so it is normal there are some hiccups.

zonca commented 1 year ago

@zkromerUCD ok, JupyterHub is back online. The isssue on the volumes was due to an outage Jetstream had over the weekend, they were able to clear the errors on the volumes.

zonca commented 1 year ago

ok, also the database and the backup system are working properly