"ValueError: Invalid empty packet received" when launched via gunicorn/gevent

[dhowe]

Not fully sure this is a bug or a configuration issue. Everything works when I run the client and server locally, but on Digital Ocean I get the errors below. The error suggests using eventlet, but I get a whole different set of errors when I use it (and this is also not what is suggested in your guide.

As suggested in the guide, I start the process like this:

$ pipenv run gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 -b 0.0.0.0:5050 server.main:app

Pipenv versions follow below.

SERVER:

pipenv run gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 -b 0.0.0.0:5050 server.main:app
[2023-04-14 15:29:13 +0000] [544591] [INFO] Starting gunicorn 20.1.0
[2023-04-14 15:29:13 +0000] [544591] [INFO] Listening at: http://0.0.0.0:5050 (544591)
[2023-04-14 15:29:13 +0000] [544591] [INFO] Using worker: geventwebsocket.gunicorn.workers.GeventWebSocketWorker
[2023-04-14 15:29:13 +0000] [544592] [INFO] Booting worker with pid: 544592
2023-04-14 15:29:15,705 {root} [INFO] logger config=beetles/logging.json
2023-04-14 15:29:38,712 {-server-} [INFO] Loaded model "gpt-4" live=False
2023-04-14 15:29:39,874 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:39] "GET / HTTP/1.1" 200 11026 1.168152
2023-04-14 15:29:40,130 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:40] "GET /static/css/main.css HTTP/1.1" 304 216 0.013334
2023-04-14 15:29:40,215 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:40] "GET /static/js/lib/jquery.min.js HTTP/1.1" 304 221 0.002203
2023-04-14 15:29:40,286 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:40] "GET /static/js/ui.js HTTP/1.1" 304 213 0.001739
2023-04-14 15:29:40,370 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:40] "GET /static/js/main.js HTTP/1.1" 304 215 0.001766
2023-04-14 15:29:40,374 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:40] "GET /static/img/dytiscus32.png HTTP/1.1" 304 221 0.002841
2023-04-14 15:29:40,555 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:40] "POST /update HTTP/1.1" 200 2390 0.002267
2023-04-14 15:29:40,568 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:40] "GET /static/img/favicon-32x32.png HTTP/1.1" 304 224 0.001752
2023-04-14 15:29:54,630 {geventwebsocket.handler} [INFO] [IP_ADDR] - - [2023-04-14 15:29:54] "GET /socket.io/?transport=polling&EIO=4&t=1681486194.32205 HTTP/1.1" 200 254 0.001019
Traceback (most recent call last):
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/gevent/pywsgi.py", line 999, in handle_one_response
    self.run_application()
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/geventwebsocket/handler.py", line 75, in run_application
    self.run_websocket()
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/geventwebsocket/handler.py", line 52, in run_websocket
    list(self.application(self.environ, lambda s, h, e=None: []))
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/flask/app.py", line 2551, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/flask_socketio/__init__.py", line 43, in __call__
    return super(_SocketIOMiddleware, self).__call__(environ,
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/engineio/middleware.py", line 63, in __call__
    return self.engineio_app.handle_request(environ, start_response)
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/socketio/server.py", line 607, in handle_request
    return self.eio.handle_request(environ, start_response)
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/engineio/server.py", line 410, in handle_request
    packets = socket.handle_get_request(
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/engineio/socket.py", line 103, in handle_get_request
    return getattr(self, '_upgrade_' + transport)(environ,
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/engineio/socket.py", line 158, in _upgrade_websocket
    return ws(environ, start_response)
  File "/home/dhowe/.local/share/virtualenvs/Beetlemania-ygLvHPii/lib/python3.10/site-packages/engineio/async_drivers/eventlet.py", line 16, in __call__
    raise RuntimeError('You need to use the eventlet server. '
RuntimeError: You need to use the eventlet server. See the Deployment section of the documentation for more information.
2023-04-14T15:29:54Z {'REMOTE_ADDR': '[IP_ADDR]', 'REMOTE_PORT': '57451', 'HTTP_HOST': '[OCEAN_IP]:5050', (hidden keys: 25)} failed with RuntimeError

CLIENT:

Traceback (most recent call last):
  File "/Users/dhowe/git/prompt-gen-client/socketio-test.py", line 27, in <module>
    sio.connect('ws://192.241.209.27:5050', auth={'uid': uid})
  File "/Users/dhowe/.local/share/virtualenvs/prompt-gen-client-McpHGEqz/lib/python3.11/site-packages/socketio/client.py", line 331, in connect
    self.eio.connect(real_url, headers=real_headers,
  File "/Users/dhowe/.local/share/virtualenvs/prompt-gen-client-McpHGEqz/lib/python3.11/site-packages/engineio/client.py", line 191, in connect
    return getattr(self, '_connect_' + self.transports[0])(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dhowe/.local/share/virtualenvs/prompt-gen-client-McpHGEqz/lib/python3.11/site-packages/engineio/client.py", line 330, in _connect_polling
    if self._connect_websocket(url, headers, engineio_path):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dhowe/.local/share/virtualenvs/prompt-gen-client-McpHGEqz/lib/python3.11/site-packages/engineio/client.py", line 458, in _connect_websocket
    pkt = packet.Packet(encoded_packet=p)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dhowe/.local/share/virtualenvs/prompt-gen-client-McpHGEqz/lib/python3.11/site-packages/engineio/packet.py", line 27, in __init__
    self.decode(encoded_packet)
  File "/Users/dhowe/.local/share/virtualenvs/prompt-gen-client-McpHGEqz/lib/python3.11/site-packages/engineio/packet.py", line 52, in decode
    raise ValueError('Invalid empty packet received')
ValueError: Invalid empty packet received

Process finished with exit code 1

Pipenv versions

addict==2.4.0
aiohttp==3.8.4
aiosignal==1.3.1
amqp==5.1.1
asn1crypto==1.5.1
astroid==2.15.2
async-timeout==4.0.2
attrs==22.2.0
backoff==2.2.1
bcrypt==4.0.1
bidict==0.22.1
billiard==3.6.4.0
blobfile==2.0.1
celery==5.2.7
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==3.1.0
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.6
contourpy==1.0.7
cryptography==39.0.2
cycler==0.11.0
dill==0.3.6
dnspython==2.3.0
-e git+ssh://git@github.com/openai/evals.git@5c5cd8e2e79ebc8c991bfd1ca056ba28172b3d4c#egg=evals
eventlet==0.33.3
filelock==3.10.0
fire==0.5.0
Flask==2.2.3
Flask-Cors==3.0.10
Flask-SocketIO==5.3.3
Flask-SQLAlchemy==3.0.3
fonttools==4.39.2
frozenlist==1.3.3
gevent==22.10.2
gevent-websocket==0.10.1
greenlet==2.0.2
gunicorn==20.1.0
h11==0.14.0
idna==3.4
isort==5.12.0
itsdangerous==2.1.2
Jinja2==3.1.2
joblib==1.2.0
jsonpickle==3.0.1
kiwisolver==1.4.4
kombu==5.2.4
langdetect==1.0.9
lazy-object-proxy==1.9.0
lxml==4.9.2
lz4==4.3.2
MarkupSafe==2.1.2
matplotlib==3.7.1
mccabe==0.7.0
mock==5.0.1
multidict==6.0.4
mypy==1.1.1
mypy-extensions==1.0.0
nltk==3.8.1
numpy==1.24.2
openai==0.27.4
oscrypto==1.3.0
packaging==23.0
pandas==1.5.3
Pillow==9.4.0
platformdirs==3.2.0
portalocker==2.7.0
prompt-toolkit==3.0.38
pyarrow==10.0.1
pycparser==2.21
pycryptodomex==3.17
pydantic==1.10.6
PyJWT==2.6.0
pylint==2.17.2
pyOpenSSL==23.0.0
pyparsing==3.0.9
python-dateutil==2.8.2
python-engineio==4.4.0
python-socketio==5.8.0
pytz==2023.3
PyYAML==6.0
pyzstd==0.15.4
raven==6.10.0
redis==4.5.4
regex==2023.3.23
requests==2.28.2
sacrebleu==2.3.1
simple-websocket==0.10.0
six==1.16.0
snowflake-connector-python==3.0.1
SQLAlchemy==2.0.5.post1
tabulate==0.9.0
termcolor==2.2.0
tiktoken==0.3.3
tomli==2.0.1
tomlkit==0.11.7
tqdm==4.65.0
typing_extensions==4.5.0
urllib3==1.26.15
vine==5.0.0
waitress==2.1.2
wcwidth==0.2.6
Werkzeug==2.2.3
wrapt==1.15.0
wsproto==1.2.0
yarl==1.8.2
zope.event==4.6
zope.interface==6.0

[miguelgrinberg]

The link that you included in your question explains this.

The simplest deployment strategy is to start the web server by calling socketio.run(app) as shown in examples above. This will look through the packages that are installed for the best available web server start the application on it. The current web server choices that are evaluated are eventlet, gevent and the Flask development server.

Do you see? You are running gevent as web server instead of letting Flask-SocketIO pick the best choice. But then you aren't explicitly telling Flask-SocketIO to work with gevent, so the package looks at what you have installed in the order of precendence stated above. It finds eventlet in your venv, so it assumes that's what you are using.

Why do you have eventlet and gevent both installed? These are competing packages, you will never use both, it's one or the other. Pick the one you want, uninstall the other and then let's see if there are any errors left.

[dhowe]

Thanks @miguelgrinberg! To answer your questions, I was using socketio.app for development. But for production wanted to use gunicorn/ gevent. I thought I only installed eventlet later in response to the error message.... but perhaps I had inadvertently installed it earlier, as things seem to work now that I've uninstalled it. Much appreciated...