PyPy Support (with patch)

[GoogleCodeExporter]

Attempting to install regex 2015.03.18 under PyPy (tested with 2.5 and 2.6) 
produces the following compilation warnings, but appears to work:

    Python2/_regex.c:18674:19: warning: implicit declaration of function 'PyLong_FromUnicode' is invalid in C99 [-Wimplicit-function-declaration]
            int_obj = PyLong_FromUnicode(characters, length, 0);
                      ^
    Python2/_regex.c:18674:17: warning: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
            int_obj = PyLong_FromUnicode(characters, length, 0);
                    ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Naturally, though, importing the regex module fails due to the missing function:

    Python 2.7.9 (58059c786617, Mar 15 2015, 23:01:32)
    [PyPy 2.6.0-alpha0 with GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
    >>>> import regex
    Traceback (most recent call last):
          File "<stdin>", line 1, in <module>
          File "//site-packages/regex.py", line 392, in <module>
            import _regex_core
          File "//site-packages/_regex_core.py", line 21, in <module>
            import _regex
           ImportError: unable to load extension module '//site-packages/_regex.pypy-26.so':  dlopen(//site-packages/_regex.pypy-26.so, 6): Symbol not found: _PyLong_FromUnicode
    Referenced from: //site-packages/_regex.pypy-26.so
    Expected in: flat namespace
         in //site-packages/_regex.pypy-26.so

Fortunately, `PyLong_FromUnicode` is a simple wrapper around other functions 
and can easily be defined. I got regex to compile, install, and run all the 
examples I tried by including this code fragment (based on the CPython 
implementation) in `_regex.c`:

    #ifdef PYPY_VERSION
    /* PyPy does not defive PyLong_FromUnicode, so include our own implementation */
    PyObject *
    PyLong_FromUnicode(Py_UNICODE *u, Py_ssize_t length, int base)
    {
        PyObject *result;
        char *buffer = (char *)PyMem_MALLOC(length+1);

        if (buffer == NULL)
            return NULL;

        if (PyUnicode_EncodeDecimal(u, length, buffer, NULL)) {
            PyMem_FREE(buffer);
            return NULL;
        }
        result = PyLong_FromString(buffer, NULL, base);
        PyMem_FREE(buffer);
        return result;
    }
    #endif

Is there any chance of getting something like this in the code base? I'd be 
happy to help further with testing or whatever else is needed.

Thank you!

Original issue reported on code.google.com by jason.ma...@nextthought.com on 27 Mar 2015 at 3:45

[GoogleCodeExporter]

It could be argued that it's a fault of PyPy.

Is that just for Python 2.5-2.7?

Original comment by re...@mrabarnett.plus.com on 27 Mar 2015 at 5:40

[GoogleCodeExporter]

PyPy's support for the CPython API is known to be incomplete; being based on a 
completely different model than CPython makes it difficult to implement the 
full API. It tends to get better with each release, but the preferred way to 
make use of C extensions under PyPy is via the CFFI interfaces---this allows 
the PyPy JIT full optimization opportunities. Obviously that's a bigger change.

That said, I don't know why the PyLong_FromUnicode API is not implemented 
(perhaps because it's such a simple wrapper?). There is this bug on the issue 
tracker indicating that it's not supported: 
https://bitbucket.org/pypy/pypy/issue/1428/

I did only build and test this under the Python2 version of PyPy (the current 
version supports 2.7.9). PyPy does have a Python3 version that supports the 
3.2.5 API, and it demonstrates the same problem. A similar patch would probably 
work there, but since PyLong_FromUnicode is deprecated in 3.3 (and PyPy is 
expecting to produce a 3.4 compatible version soon),  I wasn't sure what would 
be best.

Original comment by jason.ma...@nextthought.com on 27 Mar 2015 at 5:54

[GoogleCodeExporter]

My code uses PyLong_FromUnicode for Python 2.5-2.7 and Python 3.1-3.2 and 
PyLong_FromUnicodeObject for Python 3.3+. Does PyPy support 
PyLong_FromUnicodeObject for Python 3.3+?

Original comment by re...@mrabarnett.plus.com on 27 Mar 2015 at 7:23

[GoogleCodeExporter]

There isn't a PyPy release with support for 3.3+ yet, so I can't say 
definitively what will be supported. Looking in their source repository at what 
will become the 3.3+ compatible release (hopefully I'm in the right place), 
though, I don't see any support for either of those APIs. At least not yet.

Original comment by jason.ma...@nextthought.com on 27 Mar 2015 at 7:32