python-trio / trio

Trio – a friendly Python library for async concurrency and I/O
https://trio.readthedocs.io
Other
6.21k stars 344 forks source link

`AttributeError: module 'socket' has no attribute 'AI_NUMERICSERV'. Did you mean: 'NI_NUMERICSERV'?` #3133

Open barracuda156 opened 1 week ago

barracuda156 commented 1 week ago

AI_NUMERICSERV may not be defined on a given platform. However it is used unconditionally, which fails in that case.

Example of the problem:

36-63% /opt/local/Library/Frameworks/Python.framework/Versions/3.12/bin/yt
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/bin/yt", line 5, in <module>
    from mps_youtube import main
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/mps_youtube/main.py", line 36, in <module>
    completer = util.CommandCompleter()
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/mps_youtube/util.py", line 586, in __init__
    from . import config
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/mps_youtube/config.py", line 12, in <module>
    import pylast
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pylast/__init__.py", line 38, in <module>
    import httpx
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/__init__.py", line 2, in <module>
    from ._api import *
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_api.py", line 6, in <module>
    from ._client import Client
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 30, in <module>
    from ._transports.asgi import ASGITransport
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_transports/__init__.py", line 3, in <module>
    from .default import *
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_transports/default.py", line 33, in <module>
    import httpcore
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/__init__.py", line 1, in <module>
    from ._api import request, stream
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_api.py", line 5, in <module>
    from ._sync.connection_pool import ConnectionPool
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_sync/__init__.py", line 1, in <module>
    from .connection import HTTPConnection
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 12, in <module>
    from .._synchronization import Lock
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_synchronization.py", line 11, in <module>
    import trio
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/__init__.py", line 25, in <module>
    from . import abc, from_thread, lowlevel, socket, to_thread
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/socket.py", line 18, in <module>
    from . import _socket
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_socket.py", line 169, in <module>
    _NUMERIC_ONLY = _stdlib_socket.AI_NUMERICHOST | _stdlib_socket.AI_NUMERICSERV
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'socket' has no attribute 'AI_NUMERICSERV'. Did you mean: 'NI_NUMERICSERV'?

In C code this works fine:

#ifndef AI_NUMERICSERV
#define AI_NUMERICSERV 0
#endif

What can be done here?

CoolCat467 commented 1 week ago

I am curious about what platform you are running Python under. Looks like MacOS at first glance, but full details would be helpful. Is there another module named socket that is shadowing the built-in version? I'm not seeing anything immediately about AI_NUMERICSERV being platform-specific.

barracuda156 commented 1 week ago

Yeah, that is MacOS, but a pretty old one, a developer build of 10.6 on powerpc ) AI_NUMERICSERV is not defined in that macOS SDK (it appears in 10.6.x, I believe, released versions, so 10.5.8 on Intel won’t have it either).

(I am not really sure re other modules with socket name and how to check that.)

Simply deleting _stdlib_socket.AI_NUMERICSERV fixes the issue, at least yewtube app works then. However deleting it, even conditionally, would be a bad fix. But I do not know how to use it conditionally in Python code. Is there something like #ifdef AI_NUMERICSERV?

CoolCat467 commented 1 week ago

Kind of, in Python you would do something like this:

_NUMERIC_ONLY = _stdlib_socket.AI_NUMERICHOST
_NUMERIC_ONLY |= getattr(_stdlib_socket, "AI_NUMERICSERV", 0)

where it tries to get attribute AI_NUMERICSERV and bitwise OR that with what is already in _NUMERIC_ONLY, but OR with 0 if attribute does not exist.

It might not be safe to do so, but you could also replace AI_NUMERICSERV variable with the actual value it holds, being 1024 at least on my machine, but if the variable doesn't exist the host machine probably doesn't handle it.

barracuda156 commented 1 week ago

@CoolCat467 Awesome, this works. Could we fix this in the master? I can make a PR referring to you.

CoolCat467 commented 1 week ago

Sure, I don't see why not

A5rocks commented 1 week ago

Heh, looking this up I find https://github.com/nodejs/node-v0.x-archive/issues/6. Seems defaulting to 0 is OK

barracuda156 commented 1 week ago

@A5rocks It has been used in a number of projects. For example, in MPICH's libfabric: https://github.com/ofiwg/libfabric/pull/8045

jakkdl commented 1 week ago

Looking it up more, these versions look absolutely ancient? 10.5 ended support in 2012 and 10.6 ended support in 2014. We generally don't care to support systems that old, and if I'd seen a fix in the codebase for a system with 10+ year since EOL I would probably vote for its removal. (Looks like it's also not available on glibc <= 2.3.0, but that's 22 years old.)

So I'm somewhat confused why you need to support such an old SDK, and that other programs are working. You might be better off with something like https://github.com/macports/macports-legacy-support to patch the C libraries themself.

It's not a big problem doing a getattr, but I'm generally against expecting developers to support ancient systems. Unless you're aware of a larger set of people still using this, I think perhaps you should just set the value yourself in some way or another.

barracuda156 commented 1 week ago

@jakkdl Well, first of all it is just the right thing to do, when a flag is not universally supported. While my motivation is macOS, it is pretty likely other platforms are affected too, even though such may be very few. When it is about a single line of code, the cost of having a better code is zero.

Secondly, there are a lot of people using those legacy macOS, since those are the last to run on PowerPC hardware. You could check activity on MacRumors subforum for PowerPC Macs, for example, as an evidence. Specifically yewtube (which depends on trio) was of interest, and someone asked to add it to MacPorts. I can confirm that the app actually works on PowerPC after this tiny fix: search, streaming and downloading. Again, at the cost of a single extra line why not improve user experience, even if it is about two dozens of users and not thousands?

P. S. Going with legacysupport is suboptimal, even if feasible (I am not even sure it can be picked somehow from Python runtime and not aware of examples). That will leave the code broken outside of MacPorts (and NetBSD’s pkgsrc and Fink also have some support for older macOS) and introduce an unneeded dependency (just for the sake of defining that flag to 0).