search

pypa / setuptools

Official project repository for the Setuptools build system
https://pypi.org/project/setuptools/
MIT License
2.5k stars 1.19k forks source link

[BUG] build with --py-limited-api does not link library with -lpython3 on Windows with mingw32-x64 #4224

Open genotrance opened 7 months ago

genotrance commented 7 months ago

setuptools version

69.0.3

Python version

3.12

OS

Windows

Additional environment information

No response

Description

Using cffi to generate a wrapper for libcurl. On Windows, cffi links with -lpython312 for mingw and so the build does not work on other versions of Python.

gcc -shared -s build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\release\_libcurl_cffi.o build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\release\_libcurl_cffi.def -LE:\libs\x86_64-windows -LC:\Users\user\AppData\Local\Temp\Mxt236\var\log\xwin\build-env-ywcrviwm\libs -LC:\Users\user\scoop\apps\python\current\libs -LC:\Users\user\scoop\apps\python\current -LC:\Users\user\AppData\Local\Temp\Mxt236\var\log\xwin\build-env-ywcrviwm\PCbuild\amd64 -lcurl -lpython312 -lvcruntime140 -o build\lib.win-amd64-cpython-312\_libcurl_cffi.pyd

Manually changing the -lpython312 to -lpython3 fixes the issue. I'm not sure where this command gets generated - opened issue with cffi who redirected to setuptools.

Python build command line I used is as follows:

python3 -m build -w . -C="--build-option=build" -C="--build-option=--compiler" -C="--build-option=mingw32" -C="--build-option=bdist_wheel" -C="--build-option=--py-limited-api" -C="--build-option=cp32"

This works fine on Linux and OSX which don't need the --compiler=mingw32 flag but link with -lpython3, making the wheels generated functional with all versions of Python since 3.2.

Expected behavior

setuptools should link with -lpython3 when using --py-limited-api with mingw32.

How to Reproduce

On Windows with mingw32-x64 installed (scoop install mingw):

git clone https://github.com/genotrance/mcurl
cd mcurl
bash build.sh

Will get a wheel created which will only run on the Python version that was used to build the wheel.

Output

...
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-312
creating build\lib.win-amd64-cpython-312\mcurl
copying mcurl\gen.py -> build\lib.win-amd64-cpython-312\mcurl
copying mcurl\__init__.py -> build\lib.win-amd64-cpython-312\mcurl
running egg_info
writing mcurl.egg-info\PKG-INFO
writing dependency_links to mcurl.egg-info\dependency_links.txt
writing requirements to mcurl.egg-info\requires.txt
writing top-level names to mcurl.egg-info\top_level.txt
reading manifest file 'mcurl.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE.txt'
writing manifest file 'mcurl.egg-info\SOURCES.txt'
copying mcurl\cacert.pem -> build\lib.win-amd64-cpython-312\mcurl
running build_ext
generating cffi module 'build\\temp.win-amd64-cpython-312\\Release\\_libcurl_cffi.c'
creating build\temp.win-amd64-cpython-312
creating build\temp.win-amd64-cpython-312\Release
building '_libcurl_cffi' extension
creating build\temp.win-amd64-cpython-312\Release\build
creating build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312
creating build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\release
gcc -mdll -O -Wall -DCURL_DISABLE_DEPRECATION -IE:\Dropbox\Private\github\mcurl\mcurllib\x86_64-windows/dl/LibCURL/include/curl -IC:\Users\gt\AppData\Local\Temp\Mxt236\var\log\xwin\build-env-ywcrviwm\include -IC:\Users\gt\scoop\apps\python\current\include -IC:\Users\gt\scoop\apps\pytho
n\current\Include -c build\temp.win-amd64-cpython-312\Release\_libcurl_cffi.c -o build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\release\_libcurl_cffi.o
writing build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\release\_libcurl_cffi.def
gcc -shared -s build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\release\_libcurl_cffi.o build\temp.win-amd64-cpython-312\Release\build\temp.win-amd64-cpython-312\release\_libcurl_cffi.def -LE:\Dropbox\Private\github\mcurl\mcurllib\x86_64-windows -LC:\Users\gt\A
ppData\Local\Temp\Mxt236\var\log\xwin\build-env-ywcrviwm\libs -LC:\Users\gt\scoop\apps\python\current\libs -LC:\Users\gt\scoop\apps\python\current -LC:\Users\gt\AppData\Local\Temp\Mxt236\var\log\xwin\build-env-ywcrviwm\PCbuild\amd64 -lcurl -lpython312 -lvcruntime140 -o build\lib.win-am
d64-cpython-312\_libcurl_cffi.pyd
installing to build\bdist.win-amd64\wheel
running install
running install_lib
creating build\bdist.win-amd64
creating build\bdist.win-amd64\wheel
creating build\bdist.win-amd64\wheel\mcurl
copying build\lib.win-amd64-cpython-312\mcurl\cacert.pem -> build\bdist.win-amd64\wheel\.\mcurl
copying build\lib.win-amd64-cpython-312\mcurl\gen.py -> build\bdist.win-amd64\wheel\.\mcurl
copying build\lib.win-amd64-cpython-312\mcurl\__init__.py -> build\bdist.win-amd64\wheel\.\mcurl
copying build\lib.win-amd64-cpython-312\_libcurl_cffi.pyd -> build\bdist.win-amd64\wheel\.
running install_egg_info
Copying mcurl.egg-info to build\bdist.win-amd64\wheel\.\mcurl-8.6.0.0-py3.12.egg-info
running install_scripts
creating build\bdist.win-amd64\wheel\mcurl-8.6.0.0.dist-info\WHEEL
creating 'E:\Dropbox\Private\github\mcurl\dist\.tmp-kd4m1sg_\mcurl-8.6.0.0-cp32-abi3-win_amd64.whl' and adding 'build\bdist.win-amd64\wheel' to it
adding '_libcurl_cffi.pyd'
adding 'mcurl/__init__.py'
adding 'mcurl/cacert.pem'
adding 'mcurl/gen.py'
adding 'mcurl-8.6.0.0.dist-info/LICENSE.txt'
adding 'mcurl-8.6.0.0.dist-info/METADATA'
adding 'mcurl-8.6.0.0.dist-info/WHEEL'
adding 'mcurl-8.6.0.0.dist-info/top_level.txt'
adding 'mcurl-8.6.0.0.dist-info/RECORD'
removing build\bdist.win-amd64\wheel
Successfully built mcurl-8.6.0.0-cp32-abi3-win_amd64.whl
abravalheri commented 7 months ago

Hi @genotrance , as far as I remember setuptools/ distutils do not fully support mingw yet. There is a feature request in distutils that mentions that:

https://github.com/pypa/distutils/issues/34

If you are interested in such feature I believe, you can join the discussion and/or contribute to the implementation via PRs.

genotrance commented 7 months ago

@abravalheri - mingw32 works great, except for linking to the specific Python version. If I manuallly change the link command and update the wheel, it works on all Python versions. So the compile works fine, just not the link.

If this is a distutils issue, can this ticket be moved to that repo?

abravalheri commented 7 months ago

I believe that all the MingW discussions are happening in the pypa/distutils repository, so yeah, I think it would be better to investigate the issue there. Although I don't have the admin rights to manually transfer the issue, you might be interested in opening a new one there.

lazka commented 7 months ago

We currently patch this downstream like this: https://github.com/msys2/MINGW-packages/blob/10f6dec302dc204bc084baa5a68b24e1ef29405e/mingw-w64-python-setuptools/0010-support-py_limited_api.patch

Since the py-limited-api is a setuptools and not a distutils feature I think it makes sense to keep the discussion here. Unless we make distutils optionally depend on the setuptools subclass.