Monkey patch warning when using PEX to run a gunicorn app

[mariano-filipe]

Hi guys, my application is being packaged with pex through the pants build tool pex_binary directive. In previous versions of pants and pex I wasn't getting any warning in the startup of my gunicorn flask app, but in recent versions I started receiving the following warning:

MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['pex.fetcher (/tmp/pants-sandbox-me3cWc/.pex/.bootstrap/pex/fetcher.py)']. monkey.patch_all()

Searching for this issue on github I found out that I need to monkey patch as soon as possible in my app. However searching in the pex docs I haven't found any way to monkey patch earlier than my current pex entry_point. In the warning I can see that it's complaining of a file called pex/fetcher.py which I have no control over.

Do you guys have any recommendation on how to fix this issue?

[jsirois]

@mariano-filipe can you provide a small repro? That would help tremendously.

[jsirois]

Ok, I figured out the repro, but its great when you can do this when submitting a bug report, especially when it involves systems outside the one you're reporting against:

:; cat app.py
from gevent import monkey
monkey.patch_all()

from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

:; pex flask gevent gunicorn --module app -c gunicorn --inject-args app:app -o app.pex
:; ./app.pex &
[1] 46657
:; [2024-06-04 15:20:40 -0700] [46657] [INFO] Starting gunicorn 22.0.0
[2024-06-04 15:20:40 -0700] [46657] [INFO] Listening at: http://127.0.0.1:8000 (46657)
[2024-06-04 15:20:40 -0700] [46657] [INFO] Using worker: sync
[2024-06-04 15:20:40 -0700] [46728] [INFO] Booting worker with pid: 46728
/home/jsirois/dev/pex-tool/pex/app.py:2: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['pex.fetcher (/home/jsirois/.pex/unzipped_pexes/1f919528d7942b66beb34e5b56a696a21ae82673/.bootstrap/pex/fetcher.py)'].
  monkey.patch_all()

:; curl http://127.0.0.1:8000
Hello, World!

[jsirois]

The warning turns out to be a red-herring. Although pex/fetcher.py is included in the PEX .bootstrap/, it is unused; so gevent is missing the mark here. Nonetheless, the .bootstrap/ should only contain Pex code needed to boot a PEX file and so pex/fetcher.py should go.

[jsirois]

Ok, pex/fetcher.py can actually be used at runtime in the case of a --venv PEX, although a re-exec happens after that, clearing state and that use also does not call install_pip which is the only case where an URLFetcher might be used. As such, just making the import ssl lazy is enough.

[mariano-filipe]

@jsirois thanks a lot for the quick response and explanation. The new version will help us moving forward with new pants versions too. Appreaciate it 🙏🏽

[jsirois]

Ok @mariano-filipe the fix is released:

• https://github.com/pex-tool/pex/releases/tag/v2.3.2
• https://pypi.org/project/pex/2.3.2/