python-trio / trio

Trio – a friendly Python library for async concurrency and I/O
https://trio.readthedocs.io
Other
5.98k stars 325 forks source link

Trio fails under gevent with NotImplementedError: unsupported platform #3013

Open pg1671 opened 3 weeks ago

pg1671 commented 3 weeks ago

Trio is installed as a dependancy of httpx.

If I run the code:

from gevent.monkey import patch_all; patch_all()
import gevent
import httpx

def worker(n):
     print(n)

if __name__ == '__main__':
    jobs = [gevent.spawn(worker, job_no) for job_no in range(10)]
    gevent.joinall(jobs)

I get:

Traceback (most recent call last):
  File "/tests/gevent_test.py", line 3, in
    import httpx
  File "/work/env3.11/lib/python3.11/site-packages/httpx/init.py", line 2, in
    from ._api import delete, get, head, options, patch, post, put, request, stream
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_api.py", line 6, in
    from ._client import Client
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_client.py", line 32, in
    from ._transports.default import AsyncHTTPTransport, HTTPTransport
  File "/work/env3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 32, in
    import httpcore
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/init.py", line 1, in
    from ._api import request, stream
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_api.py", line 5, in
    from ._sync.connection_pool import ConnectionPool
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_sync/init.py", line 1, in
    from .connection import HTTPConnection
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 12, in
    from .._synchronization import Lock
  File "/work/env3.11/lib/python3.11/site-packages/httpcore/_synchronization.py", line 11, in
    import trio
  File "/work/env3.11/lib/python3.11/site-packages/trio/init.py", line 23, in
    from ._core import TASK_STATUS_IGNORED as TASK_STATUS_IGNORED # isort: split
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/init.py", line 21, in
    from ._local import RunVar, RunVarToken
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/_local.py", line 9, in
    from . import _run
  File "/work/env3.11/lib/python3.11/site-packages/trio/_core/_run.py", line 2840, in
    raise NotImplementedError("unsupported platform")
NotImplementedError: unsupported platform
A5rocks commented 3 weeks ago

Hi, what trio version are you using? (Just to double check, though I haven't tried to reproduce yet)

pg1671 commented 3 weeks ago

Hi .. it was the latest 0.25.1.

A5rocks commented 3 weeks ago

While I'm not sure this will fix it, maybe try pip install git+https://github.com/python-trio/trio? https://github.com/python-trio/trio/pull/2928 was merged since v0.25.1 was released

pg1671 commented 3 weeks ago

Hi .. it doesn't appear to fix it in the example code above. Still getting the same error.

pg1671 commented 3 weeks ago

I tried the same commands as the top of #2928 and got the error as well.

I am using Python 3.11.

CoolCat467 commented 3 weeks ago

What operating system are you using? If you can post the results of

python3.11 -m platform

that would be helpful.

A5rocks commented 3 weeks ago

We should probably include that in the assert message tbh.

I'm also curious about your platform because after pip install trio httpx gevent in a venv on Windows 11, I get this: (notice: no errors)

>>> from gevent.monkey import patch_all; patch_all()
True
>>> import gevent
>>> import httpx
>>>
>>> def worker(n):
...      print(n)
...
>>> if __name__ == '__main__':
...     jobs = [gevent.spawn(worker, job_no) for job_no in range(10)]
...     gevent.joinall(jobs)
...
0
1
2
3
4
5
6
7
8
9
[<Greenlet at 0x16460a8bce0: _run>, <Greenlet at 0x16460acbe20: _run>, <Greenlet at 0x16460acbec0: _run>, <Greenlet at 0x16460acbba0: _run>, <Greenlet at 0x16460acbf60: _run>, <Greenlet at 0x16460adc040: _run>, <Greenlet at 0x16460adc0e0: _run>, <Greenlet at 0x16460adc180: _run>, <Greenlet at 0x16460adc220: _run>, <Greenlet at 0x16460adc2c0: _run>]

... I assume some sort of Linux? Looking at the code, windows uses a set of primitives always, rather than trying to find something that works.

I think your error totally makes sense and we can probably figure something out (detecting gevent monkey patching?), I'm just surprised this hasn't happened before as far as I know. As a short term fix, when I pip install httpx it doesn't look like it pulls in trio, so I don't think that's a required dependency. Maybe your application will work after uninstalling trio to force httpx to use something else?

CoolCat467 commented 3 weeks ago

Because of the way the logic works, the system this is failing on is not windows, not linux, not epoll-based, and not kqueue-based. Given the what the select docs say, I predict the target system is Solaris or a Solaris derivative, which uses devpoll for polling.

CoolCat467 commented 3 weeks ago

Might not actually go anywhere because of issues with getting a solaris runner in CI, but created #3014 for attempting to add solaris support

njsmith commented 3 weeks ago

gevent monkey patches like, every io routine in the standard library. Idk what exactly the interaction is that's breaking trio but it's super unsurprising that something is breaking, because you're not really running on any of those platforms, or even the regular Python interpreter; you're running on the platform "gevent" and the language "gevent-flavored Python", with its own thread API and all kinds of things.

On Mon, Jun 10, 2024, 01:45 CoolCat467 @.***> wrote:

Might not actually go anywhere because of issues with getting a solaris runner in CI, but created #3014 https://github.com/python-trio/trio/pull/3014 for attempting to add solaris support

— Reply to this email directly, view it on GitHub https://github.com/python-trio/trio/issues/3013#issuecomment-2157731971, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEU42AEX5TMV5OIS6TEE3DZGVRSLAVCNFSM6AAAAABJBIZSVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJXG4ZTCOJXGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

pg1671 commented 3 weeks ago

Hi, the platform is ubuntu 22.04.

Here is the output from python3.11 -m platform:

Linux-6.5.0-1018-aws-x86_64-with-glibc2.35