AgnostiqHQ / covalent

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
https://www.covalent.xyz
Apache License 2.0
774 stars 92 forks source link

Covalent programmatic start hangs in Google Colab notebook #1899

Open araghukas opened 10 months ago

araghukas commented 10 months ago

@araghukas just checked this in google colab with latest pre-release, does not seem to be working 👀

Originally posted by @santoshkumarradha in https://github.com/AgnostiqHQ/covalent/issues/1854#issuecomment-1869137210

araghukas commented 10 months ago

Tried this in the default Google Colab notebook after installing covalent with !pip install --upgrade --pre covalent (Python 3.10, Covalent 0.233.1rc0)

Nothing helpful in the logs, unfortunately

!covalent start -d > /dev/null & tail -f ~/.cache/covalent/covalent_ui.log
[2024-01-05 03:26:59,907] [INFO] Will watch for changes in these directories: ['/usr/local/lib/python3.10/dist-packages/covalent_ui']
[2024-01-05 03:26:59,909] [INFO] Uvicorn running on http://localhost:48008/ (Press CTRL+C to quit)
[2024-01-05 03:26:59,910] [INFO] Started reloader process [8862] using WatchFiles
[2024-01-05 03:27:06,227] [INFO] Started server process [8956]
[2024-01-05 03:27:06,227] [INFO] Waiting for application startup.
[2024-01-05 03:27:06,228] [DEBUG] runner_ng.py: Line 348 in _listen_for_job_events: Starting event listener
[2024-01-05 03:27:06,228] [DEBUG] dispatcher.py: Line 498 in _node_event_listener: Starting event listener
[2024-01-05 03:27:06,228] [INFO] Application startup complete.
[2024-01-05 03:27:07,003] [INFO] 127.0.0.1:44228 - "GET / HTTP/1.1" 200

(logs hang at this point.)

for comparison, I tried the above on my laptop as well:

$ covalent start -d > /dev/null & tail -f ~/.cache/covalent/covalent_ui.log
[2024-01-05 03:29:37,189] [INFO] Will watch for changes in these directories: ['/Users/araghukas/miniconda3/envs/funcserve/lib/python3.9/site-packages/covalent_ui']
[2024-01-05 03:29:37,192] [INFO] Uvicorn running on http://localhost:48008 (Press CTRL+C to quit)
[2024-01-05 03:29:37,192] [INFO] Started reloader process [93693] using WatchFiles
[2024-01-05 03:29:39,073] [INFO] Started server process [93713]
[2024-01-05 03:29:39,073] [INFO] Waiting for application startup.
[2024-01-05 03:29:39,074] [DEBUG] runner_ng.py: Line 348 in _listen_for_job_events: Starting event listener
[2024-01-05 03:29:39,074] [DEBUG] dispatcher.py: Line 498 in _node_event_listener: Starting event listener
[2024-01-05 03:29:39,074] [INFO] Application startup complete.
[2024-01-05 03:29:39,967] [INFO] 127.0.0.1:57296 - "GET / HTTP/1.1" 200
[1]  + 93689 done       covalent start -d > /dev/null
[2024-01-05 03:29:40,483] [DEBUG] = connection is CONNECTING
[2024-01-05 03:29:40,484] [DEBUG] < GET /socket.io/?EIO=4&transport=websocket HTTP/1.1
[2024-01-05 03:29:40,484] [DEBUG] < host: localhost:48008
[2024-01-05 03:29:40,484] [DEBUG] < connection: Upgrade
[2024-01-05 03:29:40,484] [DEBUG] < pragma: no-cache
[2024-01-05 03:29:40,484] [DEBUG] < cache-control: no-cache
...

other notes: