search

petals-infra / health.petals.dev

🏥 Health monitor for a Petals swarm
https://health.petals.dev
32 stars 26 forks source link

Issue: Resource temporarily unavailable #10

Open filopedraz opened 1 year ago

filopedraz commented 1 year ago

Getting error when starting the Flask webserver.

Here the logs:

❯ flask run --host=0.0.0.0 --port=5000
Aug 30 11:19:09.074 [INFO] Connecting to DHT
Aug 30 11:19:14.155 [INFO] Starting Flask app
Aug 30 11:19:14.159 [INFO] Starting updater
Aug 30 11:19:14.510 [ERROR] [hivemind.dht.dht._run:131] [Errno 35] Resource temporarily unavailable
Traceback (most recent call last):
  File "/Users/filippopedrazzini/Documents/Work.nosync/prem/health.petals.dev/venv/lib/python3.10/site-packages/hivemind/dht/dht.py", line 129, in _run
    method, args, kwargs = self._inner_pipe.recv()
  File "/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 421, in _recv_bytes
    return self._recv(size)
  File "/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
BlockingIOError: [Errno 35] Resource temporarily unavailable
Aug 30 11:19:17.510 [ERROR] [hivemind.dht.dht._run:131] [Errno 35] Resource temporarily unavailable
Traceback (most recent call last):
  File "/Users/filippopedrazzini/Documents/Work.nosync/prem/health.petals.dev/venv/lib/python3.10/site-packages/hivemind/dht/dht.py", line 129, in _run
    method, args, kwargs = self._inner_pipe.recv()
  File "/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 421, in _recv_bytes
    return self._recv(size)
  File "/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
BlockingIOError: [Errno 35] Resource temporarily unavailable

I followed the installation instructions provided in the README.md

I am using python=3.10 and I have a Mac M1

borzunov commented 1 year ago

Hi @filopedraz,

This seems to be a macOS compatibility issue. Please run the health monitor in Docker or on a Linux machine until it's fixed.

borzunov commented 11 months ago

For the record: I've met the same error on Linux as well, but it's very rare (happened once in a few months during a routine state update on http://health.petals.dev). This seems to be a hivemind.DHT bug that is worth deeper investigation.