Doesn't build on python 3.11

[llimllib]

thriftpy2 can't be installed from pypi against python3.11, because it expects the longintrepr.h header, which was removed in: https://github.com/python/cpython/pull/28968

Apparently this was fixed in cython here: https://github.com/cython/cython/pull/4428

On my system, the result of pip install thriftpy2 while using an asdf-installed python 3.11 is:

build output

``` $ pip install thriftpy2 Collecting thriftpy2 Using cached thriftpy2-0.4.14.tar.gz (361 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: ply<4.0,>=3.4 in ./.asdf/installs/python/3.11.0/lib/python3.11/site-packages (from thriftpy2) (3.11) Installing collected packages: thriftpy2 DEPRECATION: thriftpy2 is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559 Running setup.py install for thriftpy2 ... error error: subprocess-exited-with-error × Running setup.py install for thriftpy2 did not run successfully. │ exit code: 1 ╰─> [124 lines of output] running install /Users/llimllib/.asdf/installs/python/3.11.0/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build creating build/lib.macosx-13.0-arm64-cpython-311 creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/server.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/hook.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/thrift.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/rpc.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/utils.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/tornado.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/http.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 copying thriftpy2/_compat.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2 creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport copying thriftpy2/transport/_ssl.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport copying thriftpy2/transport/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport copying thriftpy2/transport/sslsocket.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport copying thriftpy2/transport/socket.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport copying thriftpy2/transport/base.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol copying thriftpy2/protocol/binary.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol copying thriftpy2/protocol/compact.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol copying thriftpy2/protocol/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol copying thriftpy2/protocol/exc.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol copying thriftpy2/protocol/multiplex.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol copying thriftpy2/protocol/json.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol copying thriftpy2/protocol/base.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/parser copying thriftpy2/parser/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/parser copying thriftpy2/parser/parser.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/parser copying thriftpy2/parser/exc.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/parser copying thriftpy2/parser/lexer.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/parser creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib copying thriftpy2/contrib/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/memory copying thriftpy2/transport/memory/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/memory creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/buffered copying thriftpy2/transport/buffered/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/buffered creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/framed copying thriftpy2/transport/framed/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/framed creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio copying thriftpy2/contrib/aio/server.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio copying thriftpy2/contrib/aio/client.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio copying thriftpy2/contrib/aio/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio copying thriftpy2/contrib/aio/processor.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio copying thriftpy2/contrib/aio/rpc.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio copying thriftpy2/contrib/aio/socket.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/tracking copying thriftpy2/contrib/tracking/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/tracking copying thriftpy2/contrib/tracking/tracker.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/tracking creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/transport copying thriftpy2/contrib/aio/transport/buffered.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/transport copying thriftpy2/contrib/aio/transport/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/transport copying thriftpy2/contrib/aio/transport/base.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/transport copying thriftpy2/contrib/aio/transport/framed.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/transport creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/protocol copying thriftpy2/contrib/aio/protocol/binary.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/protocol copying thriftpy2/contrib/aio/protocol/compact.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/protocol copying thriftpy2/contrib/aio/protocol/__init__.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/protocol copying thriftpy2/contrib/aio/protocol/base.py -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/aio/protocol running egg_info writing thriftpy2.egg-info/PKG-INFO writing dependency_links to thriftpy2.egg-info/dependency_links.txt writing requirements to thriftpy2.egg-info/requires.txt writing top-level names to thriftpy2.egg-info/top_level.txt reading manifest file 'thriftpy2.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'thriftpy2.egg-info/SOURCES.txt' /Users/llimllib/.asdf/installs/python/3.11.0/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'thriftpy2.protocol.cybin' as data is deprecated, please list it in `packages`. !! ############################ # Package would be ignored # ############################ Python recognizes 'thriftpy2.protocol.cybin' as an importable package, but it is not listed in the `packages` configuration of setuptools. 'thriftpy2.protocol.cybin' has been automatically added to the distribution only because it may contain data files, but this behavior is likely to change in future versions of setuptools (and therefore is considered deprecated). Please make sure that 'thriftpy2.protocol.cybin' is included as a package by using the `packages` configuration field or the proper discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" and "data files" on setuptools documentation page. !! check.warn(importable) copying thriftpy2/transport/cybase.c -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport copying thriftpy2/transport/cybase.pxd -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport copying thriftpy2/transport/cybase.pyx -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport creating build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol/cybin copying thriftpy2/protocol/cybin/cybin.c -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol/cybin copying thriftpy2/protocol/cybin/cybin.pyx -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol/cybin copying thriftpy2/protocol/cybin/endian_port.h -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/protocol/cybin copying thriftpy2/transport/memory/cymemory.c -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/memory copying thriftpy2/transport/memory/cymemory.pyx -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/memory copying thriftpy2/transport/buffered/cybuffered.c -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/buffered copying thriftpy2/transport/buffered/cybuffered.pyx -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/buffered copying thriftpy2/transport/framed/cyframed.c -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/framed copying thriftpy2/transport/framed/cyframed.pyx -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/transport/framed copying thriftpy2/contrib/tracking/tracking.thrift -> build/lib.macosx-13.0-arm64-cpython-311/thriftpy2/contrib/tracking running build_ext building 'thriftpy2.transport.cybase' extension creating build/temp.macosx-13.0-arm64-cpython-311 creating build/temp.macosx-13.0-arm64-cpython-311/thriftpy2 creating build/temp.macosx-13.0-arm64-cpython-311/thriftpy2/transport clang -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Users/llimllib/.asdf/installs/python/3.11.0/include/python3.11 -c thriftpy2/transport/cybase.c -o build/temp.macosx-13.0-arm64-cpython-311/thriftpy2/transport/cybase.o thriftpy2/transport/cybase.c:209:12: fatal error: 'longintrepr.h' file not found #include "longintrepr.h" ^~~~~~~~~~~~~~~ 1 error generated. error: command '/usr/bin/clang' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure × Encountered error while trying to install package. ╰─> thriftpy2 note: This is an issue with the package mentioned above, not pip. ```

[llimllib]

It's possible just rebuilding the C files with an updated cython will help? But I scanned your readme and I don't see how to do that, and the C files aren't in source control anyway so I can't really file a PR

[ethe]

Yes it is, I think it should be a Cython usage problem, I will try to fix it later

[ethe]

Hi, I checked Cython==0.29.32 under Python 3.11.0 and it works fine to me, maybe you could reinstall Cython to latest and reinstall thriftpy to fix this problem

[aisk]

Hi, I checked Cython==0.29.32 under Python 3.11.0 and it works fine to me, maybe you could reinstall Cython to latest and reinstall thriftpy to fix this problem

I think we should add a pyproject.toml to specify cython version in build (install) process.

[leoredi]

I have Cython==0.29.32 but I still see the same proble:

(root-vanilla) ➜  hdf2root git:(master) ✗ pip list
Package                       Version
----------------------------- -----------
anyio                         3.6.2
appnope                       0.1.3
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
asttokens                     2.1.0
attrs                         22.1.0
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
beautifulsoup4                4.11.1
bleach                        5.0.1
cffi                          1.15.1
colorama                      0.4.6
Cython                        0.29.32
debugpy                       1.6.3
decorator                     5.1.1
defusedxml                    0.7.1
entrypoints                   0.4
executing                     1.2.0
fastjsonschema                2.16.2
flit_core                     3.8.0
idna                          3.4
importlib-metadata            5.0.0
importlib-resources           5.10.0
ipykernel                     6.17.1
ipyparallel                   8.4.1
ipython                       8.6.0
ipython-genutils              0.2.0
jedi                          0.18.1
Jinja2                        3.1.2
jsonschema                    4.17.0
jupyter_client                7.4.7
jupyter_core                  5.0.0
jupyter-server                1.23.2
jupyterlab-pygments           0.2.2
MarkupSafe                    2.1.1
matplotlib-inline             0.1.6
metakernel                    0.29.2
mistune                       2.0.4
nbclassic                     0.4.8
nbclient                      0.7.0
nbconvert                     7.2.5
nbformat                      5.7.0
nest-asyncio                  1.5.6
notebook                      6.5.2
notebook_shim                 0.2.2
numpy                         1.23.5
packaging                     21.3
pandocfilters                 1.5.0
parso                         0.8.3
pexpect                       4.8.0
pickleshare                   0.7.5
pip                           22.3.1
pkgutil_resolve_name          1.3.10
platformdirs                  2.5.2
portalocker                   2.6.0
prometheus-client             0.15.0
prompt-toolkit                3.0.32
psutil                        5.9.4
ptyprocess                    0.7.0
pure-eval                     0.2.2
pycparser                     2.21
Pygments                      2.13.0
pyparsing                     3.0.9
pyrsistent                    0.19.2
python-dateutil               2.8.2
pyzmq                         24.0.1
Send2Trash                    1.8.0
setuptools                    65.5.1
six                           1.16.0
sniffio                       1.3.0
soupsieve                     2.3.2.post1
stack-data                    0.6.1
terminado                     0.17.0
tinycss2                      1.2.1
tornado                       6.2
tqdm                          4.64.1
traitlets                     5.5.0
typing_extensions             4.4.0
wcwidth                       0.2.5
webencodings                  0.5.1
websocket-client              1.4.2
wheel                         0.38.4
xrootd                        5.5.1
zipp                          3.10.0

[ethe]

I have Cython==0.29.32 but I still see the same proble:

(root-vanilla) ➜  hdf2root git:(master) ✗ pip list
Package                       Version
----------------------------- -----------
anyio                         3.6.2
appnope                       0.1.3
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
asttokens                     2.1.0
attrs                         22.1.0
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
beautifulsoup4                4.11.1
bleach                        5.0.1
cffi                          1.15.1
colorama                      0.4.6
Cython                        0.29.32
debugpy                       1.6.3
decorator                     5.1.1
defusedxml                    0.7.1
entrypoints                   0.4
executing                     1.2.0
fastjsonschema                2.16.2
flit_core                     3.8.0
idna                          3.4
importlib-metadata            5.0.0
importlib-resources           5.10.0
ipykernel                     6.17.1
ipyparallel                   8.4.1
ipython                       8.6.0
ipython-genutils              0.2.0
jedi                          0.18.1
Jinja2                        3.1.2
jsonschema                    4.17.0
jupyter_client                7.4.7
jupyter_core                  5.0.0
jupyter-server                1.23.2
jupyterlab-pygments           0.2.2
MarkupSafe                    2.1.1
matplotlib-inline             0.1.6
metakernel                    0.29.2
mistune                       2.0.4
nbclassic                     0.4.8
nbclient                      0.7.0
nbconvert                     7.2.5
nbformat                      5.7.0
nest-asyncio                  1.5.6
notebook                      6.5.2
notebook_shim                 0.2.2
numpy                         1.23.5
packaging                     21.3
pandocfilters                 1.5.0
parso                         0.8.3
pexpect                       4.8.0
pickleshare                   0.7.5
pip                           22.3.1
pkgutil_resolve_name          1.3.10
platformdirs                  2.5.2
portalocker                   2.6.0
prometheus-client             0.15.0
prompt-toolkit                3.0.32
psutil                        5.9.4
ptyprocess                    0.7.0
pure-eval                     0.2.2
pycparser                     2.21
Pygments                      2.13.0
pyparsing                     3.0.9
pyrsistent                    0.19.2
python-dateutil               2.8.2
pyzmq                         24.0.1
Send2Trash                    1.8.0
setuptools                    65.5.1
six                           1.16.0
sniffio                       1.3.0
soupsieve                     2.3.2.post1
stack-data                    0.6.1
terminado                     0.17.0
tinycss2                      1.2.1
tornado                       6.2
tqdm                          4.64.1
traitlets                     5.5.0
typing_extensions             4.4.0
wcwidth                       0.2.5
webencodings                  0.5.1
websocket-client              1.4.2
wheel                         0.38.4
xrootd                        5.5.1
zipp                          3.10.0

Whats the error message you got?

[llimllib]

It works for me now:

``` $ pip install thriftpy2 Collecting thriftpy2 Using cached thriftpy2-0.4.16.tar.gz (643 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: ply<4.0,>=3.4 in ./venv/lib/python3.11/site-packages (from thriftpy2) (3.11) Requirement already satisfied: six~=1.15 in ./venv/lib/python3.11/site-packages (from thriftpy2) (1.16.0) Installing collected packages: thriftpy2 DEPRECATION: thriftpy2 is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559 Running setup.py install for thriftpy2 ... done Successfully installed thriftpy2-0.4.16 [notice] A new release of pip available: 22.3 -> 22.3.1 [notice] To update, run: pip install --upgrade pip $ python --version Python 3.11.0 ```

[ankostis]

I'm still struggling with this on Python-3.11.2, any solution?

 /tmp/pip-build-env-osrxgf34/overlay/lib/python3.11/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'thriftpy2.protocol.cybin' is absent from the `packages` configuration.
      !!

[ankostis]

My mistake, got hooked to the warning message i pasted above, and failed to see that gcc was missing, at the bottom of the error trace:

      running build_ext
      building 'thriftpy2.transport.cybase' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/thriftpy2
      creating build/temp.linux-x86_64-cpython-311/thriftpy2/transport
      x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/python/uds-scripts.git/.venv2/include -I/usr/include/python3.11 -c thriftpy2/transport/cybase.c -o build/temp.linux-x86_64-cpython-311/thriftpy2/transport/cybase.o
      error: command 'x86_64-linux-gnu-gcc' failed: No such file or directory

To solve it, at least for those on Debian systems, run:

sudo apt install build-essential