search

aleaxit / gmpy

General Multi-Precision arithmetic for Python 2.6+/3+ (GMP, MPIR, MPFR, MPC)
https://gmpy2.readthedocs.io/en/latest/
GNU Lesser General Public License v3.0
519 stars 86 forks source link

Alternatives to using private API for conversion to/from CPython int's #467

Open skirpichev opened 8 months ago

skirpichev commented 8 months ago

Currently, we are using a bunch of private CPython functions to provide fast mpz <-> int conversion: see GMPy_MPZ_From_PyLong, GMPy_PyLong_From_MPZ and https://github.com/aleaxit/gmpy/blob/141eb88c9c4ab810d7988f0ebd17c83a6c11a8b7/src/gmpy2_convert.h#L135-L155 There should be a better way!

I see several variants to deal with this problem.

  1. The CPython 3.13 got PyLong_AsNativeBytes() and PyLong_FromNativeBytes() function to import/export arbitrary int's. We can use these functions and handle small integers case separately with PyLong_AsLong()/PyLong_FromLong(). In general, this should be much slower than the current solution. On another hand, I think that for real world scenarios - small integers case should be most important.

  2. There is a proposal to add mpz_import/export-like functions for PyLong's C API, but it seems that CPython core developers aren't very enthusiastic on this idea (see https://github.com/python/cpython/issues/111140#issuecomment-1928654497).

  3. Another possibility is the mpz_limbs_write()-like API to access unsigned bigint as an array of "digits", see this. With that kind of API on the CPython side - the gmpy2 could do fast conversion to/from int's using mpz_import/export functions.

I'm planning to open a discussion thread on the d.p.o, that will propose the 3rd option, but first I would appreciate any feedback here. CC @oscarbenjamin: perhaps this does make sense for the python-flint.

skirpichev commented 5 months ago

See https://github.com/capi-workgroup/decisions/issues/31

skirpichev commented 1 month ago

@casevh, unfortunately it seems we aren't going to have the public API on CPython side.

See https://discuss.python.org/t/pep-757-c-api-to-import-export-python-integers/63895/66

Is it does make sense for you if the PyLong_Export API will fail on some integers? How we can recover from such failures? I think such export API doesn't make sense.