Support PyPy

[mattip]

Would a PR to add PyPy to the linux, windows and macos 64 bit platforms overflow the build matrix? If not, I could try to submit one.

something like

https://bitbucket.org/pypy/pypy/downloads/pypy3.6-v7.1.1-linux64.tar.bz2 -o pypy.tar.bz2
mkdir -p pypy3
(cd pypy3; tar --strip-components=1 -xf ../pypy.tar.bz2)
export PATH=${PWD}/pypy3/bin:$PATH
echo "export PATH=${PWD}/pypy3/latest/bin:$PATH" >> $BASH_ENV
# Rebuild the _ssl module to accomodate for different openssl library
(cd pypy3/lib_pypy; pypy3 -c \
   'from _ssl_build import ffi; ffi.compile(verbose=False)' 2> /dev/null)
pypy3 -mensurepip

is what scipy uses to download and prep a pypy3 environment on circleci

[skvark]

Sorry for the slow response. I'm not very familiar with PyPy, but for example this issue suggests that PyPy is not yet compatible with the manylinux1 spec. Additionally, this repository seems to be dedicated for building PyPy wheels.

Adding PyPy to the matrix would make the maintenance burden too high since it requires (in its current state) a custom environment (at least on Linux) and most likely also a custom package index to which the wheels would be uploaded.

[extraymond]

Hi! I've tried building the package with latest pypy(7.2.0) and it works on ubuntu linux. The only changes needs to be included are:

1. Install numpy before the build script.
2. pointing pythonlib to use pypy's libpypy-c.so.

I've shared a more detail procedure on reddit here

---

Not wanting to pollute the build matrix, is it possible to upload source version to pypi and let people build on their local environment? If that's possible, I think it'll be nice to accept pypy flag for pip installs. I believe numpy is built locally for pypy since there are no wheels provided for pypy on pypi. This would allow people to not get blocked when pip install opencv-python on pypy.

[skvark]

Once https://github.com/skvark/opencv-python/issues/43 is solved (see the latest comments, some minor issues related to versioning which is determined during build), we can publish source distribution to PyPI.

[extraymond]

Thx!! That sounds nice.

[johnthagen]

@mattip Do I understand it correctly that as of PyPy 7.3.0, PyPy supports manylinux2010 wheels?

Based on the great work done in portable-pypy, the linux x86 downloads we provide are now built on top of the manylinux2010_ CentOS6 docker image. The tarballs include the needed shared objects to run on any platform that supports manylinux2010 wheels, which should include all supported versions of debian- and RedHat-based distributions (including Ubuntu, CentOS, and Fedora).

Would this make supporting PyPy3 on Linux easier? Or is this talking more about PyPy itself running on those platforms?

[Russell-Jones-OxPhys]

as an aside, see also https://morepypy.blogspot.com/2019/12/hpy-kick-off-sprint-report.html and https://github.com/pyhandle/hpy

[mattip]

This means

• PyPy now includes a vendored version of pip 19+, so pypy -mensurepip will support PyPy manylinux2010 wheels
• the PyPy you can download from bitbucket.org/pypy/pypy/downloads should run on any glibc based Linux

You still need to build PyPy-specific manylinux wheels, the cpython ones will not load into pypy

[johnthagen]

Thanks for the clarification!

[colibri17]

Any update on this?

[cole-dda]

can support pypy?, because compile too slow and too complex

[mattip]

Adding PyPy to the matrix would make the maintenance burden too high since it requires (in its current state) a custom environment (at least on Linux) and most likely also a custom package index to which the wheels would be uploaded.

No custom index would be needed. The wheels can be uploaded to PyPI just like numpy does (search for "pp37" on that page). Since this repo builds with github actions and uses actions/setup-python@v2, which supports PyPy, all that should be needed is to add pypy-3.7 to the python-version matrix

python-version: [3.6, 3.7, 3.8, 3.9, pypy-3.7]

[skvark]

Adding PyPy to the matrix would make the maintenance burden too high since it requires (in its current state) a custom environment (at least on Linux) and most likely also a custom package index to which the wheels would be uploaded.

No custom index would be needed. The wheels can be uploaded to PyPI just like numpy does (search for "pp37" on that page). Since this repo builds with github actions and uses actions/setup-python@v2, which supports PyPy, all that should be needed is to add pypy-3.7 to the python-version matrix

python-version: [3.6, 3.7, 3.8, 3.9, pypy-3.7]

Yes, that comment was written over 2 years ago when this repository was using Travis and manylinux1 and I was the sole maintainer. This repository is now maintained by the OpenCV team (https://opencv.org/opencv-python-is-now-an-official-opencv-project/). They can decide if they would like to add PyPy support to their release roadmap.

It looks like NumPy added those pre-built PyPy wheels to PyPI in version 1.19.0 (last year) which makes it now a lot easier to add PyPy support also to these packages.

[dman82499]

has anyone managed to successfully build for windows? I've tried (also setting the same cmake args to point to the correct library and am getting this error: ninja: error: 'C:UsersDevanDownloadspypy3.7-v7.3.4-win64pypy3.7-v7.3.4-win64libpypy3-c.dll', needed by 'lib/python3/cv2.cp37-win_amd64.pyd', missing and no known rule to make it.

It seems like on windows something is making ninja eliminate the parenthesis (I think it should say C:/Users/Devan/Downloads/pypy3.7-v7.3.4-win64/pypy3.7-v7.3.4-win64libpypy3-c.dll or something along the lines of that, but for some reason ninja is not putting the parenthesis in.

[mattip]

Are you specifying the location as C:/Users/Devan/Downloads/pypy3.7-v7.3.4-win64/pypy3.7-v7.3.4-win64libpypy3-c.dll or C:\Users\Devan\Downloads\pypy3.7-v7.3.4-win64\pypy3.7-v7.3.4-win64libpypy3-c.dll ?

[dman82499]

Are you specifying the location as C:/Users/Devan/Downloads/pypy3.7-v7.3.4-win64/pypy3.7-v7.3.4-win64libpypy3-c.dll or C:\Users\Devan\Downloads\pypy3.7-v7.3.4-win64\pypy3.7-v7.3.4-win64libpypy3-c.dll ?

I actually managed to fix that issue before I saw this post, but yes, that was basically the problem. I still cannot compile however, as the linker is throwing errors for lots of unresolved external symbols. This leads me to think that somehow the library still didn't register to the linker despite it now correctly appearing in the cmake enviornment variables, but I'm not sure.