Windows build comments

[casevh]

@skirpichev @oscarbenjamin

Some comments on separating GMP, MPFR, and MPC files from gmpy2. This is based on the current code and has only been tested on Windows.

Current state has the [gmp,mpfr,mpc] .h and .lib files installed in gmpy2/gmpy2 and the DLLs are installed in gmpy2.libs. I tried to create a new gmpy2*.pyd file using those installed libraries. It is much easier if the .h and .lib files are installed in gmpy2.libs.

I moved the .h and .lib files on an installed installation and I copied the src/ to a different location in the filesystem. The setup file below is sufficient to compile a new gmpy2.pyd file. Since my immediate need is to quickly compile and test, I just replaced the previously installed gmpy2*.pyd file with the new one and it worked.

Here is the modified setup.py file. I cut it down to the bare minimum.

from setuptools import Extension, setup
import sys
import os
import platform
import gmpy2

gmpy2_packagedir = os.path.dirname(gmpy2.__file__) + '.libs'

gmpy_ext = [
    Extension("gmpy2.gmpy2",
          sources=["gmpy2.c"],
              include_dirs=sys.path + [gmpy2_packagedir],
          library_dirs=sys.path + [gmpy2_packagedir],
              libraries=['mpc','mpfr','gmp'],
              extra_compile_args= ["/DSHARED=1"],
             )
           ]

setup (
       ext_modules = gmpy_ext
)

And the contents of gmpy2.libs are:

gmp.h
gmp.lib
libgmp-10.dll
mpfr.h
mpfr.lib
libmpfr-6.dll
mpc.h
mpc.lib
libmpc-3.dll
libgcc_s_seh-1.dll
libwinpthread-1.dll

Would the following strategy work for bundling the GMP, MPFR, MPC libraries? Names can easily changed.

1. Create "gmpy2libs_vN" for each version of the set of libraries. N is incremented for each new version.
2. gmpy2libs_vN contains the libraries and just enough C code to allow "import gmpy2libs_vN".
3. Users would need to import gmpy2libs_vN before loading their extension.
4. gmpy2libs_vN could contain support functions to assist detecting versions etc.

I don't understand Python packaging requirements to know if this would work or not.

[oscarbenjamin]

To me it sounds as if the strategy can work although I am unsure and I think it is discouraged by other packaging people.

What I am not sure about is how to make the linking work on linux/macos. I have seen a claim that it is impossible to make this work on macos without using a "capsule" like numpy does for its C API: https://discuss.python.org/t/linking-against-installed-extension-modules/51710

Here are other longer threads that touched on this approach (with explicit mentions of gmpy2 and python-flint in both): https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420/99 https://discuss.python.org/t/enforcing-consistent-metadata-for-packages/50008/28

I think that it is something that would be potentially worth bringing up as a packaging discussion in its own right because there is not a recommended way to make this work but other projects want this sort of thing as well.

[skirpichev]

Regarding directory layout, I don't think it's a big deal. We could offer scipy-openblas64-like interface to get right directories and library names:

>>> import scipy_openblas64
>>> scipy_openblas64.get_include_dir()
'/home/sk/src/gmpy/tmp-3.12/lib/python3.12/site-packages/scipy_openblas64/include'
>>> scipy_openblas64.get_lib_dir()
'/home/sk/src/gmpy/tmp-3.12/lib/python3.12/site-packages/scipy_openblas64/lib'
>>> scipy_openblas64.get_library()
'scipy_openblas64_'

That will be enough to fill Extension() arguments.

I'm not sure how we should declare dependency on the gmpy2libs's package (or 3, for GMP, MPFR and MPC?). This looks like an optional dependency, but rather for the build backend. I doubt such is supported.