pypy3.10 is not available

[dalcinl]

Comment:

I'm not sure this is the right feedstock to ask. Is there any technical reason for not providing pypy3.10?

[mattip]

You can see from the pypy3.8/3.9 migration (click on the "In PR" button) that there are a number of packages that currently get stuck in CI. This means that every feedstock must take the time to analyze the failures and decide whether to fix or skip the migration, which requires investment of attention. That is the scarcest resource in the conda-forge system. For instance, the h5py one is failing all tests. Someone needs to investigate those failures and see if they can be fixed. This issue around overwriting attributes has been open for a few years now without progress. Or this PR on the scikit-learn feedstock where tests time out, but seem to work on the upstream project CI.

On the other hand, there has not been an big uptake, as far as I can tell, of PyPy. Maybe I am mistaken. It is the nature of open source that feedback is not very common. Even download numbers are hard to come by.

Let's look at this the other way: is there some added benefit you could get from 3.10 that you do not currently get from 3.9?

xref https://github.com/conda-forge/pypy3.6-feedstock/issues/104

[dalcinl]

Please note I'm not interested in a full-fledged conda-forge migration to 3.10, just the ability of creating a basic environment with conda create python=3.10 pypy. Would that be possible without triggering a massive migration?

Let's look at this the other way: is there some added benefit you could get from 3.10 that you do not currently get from 3.9?

I certainly not using PyPy in production. However, I do use it in CI for regular integration tests and to test built wheels for ABI compatibility across OSs and architectures. This way I can keep my project in shape, such that surprises do not hit downstream packagers like conda-forge itself.

xref conda-forge/pypy3.6-feedstock#104

From your comments in that issue it is obvious you are quite frustrated, and rightly so. However, for those who care about keeping things tidy and thoroughly tested, having basic pypy3.10 available would be a great thing. Moreover, I would probably migrate/add manually pypy3.10 builds to my own mpi4py-feedstock (wich is currently supporting PyPy, because I care about supporting it). My project and other projects should not be held back because other projects don't follow due diligence. And you should not feel responsible of keeping all of the PyPy conda-forge ecosystem in shape. You cannot possibly watch over and go after every other project feedstock to maintain PyPy support.

[mattip]

Would that (pypy3.10) be possible without triggering a massive migration?

Yes, and thanks for the support. That might be a low-maintenance path forward.

[fbraun4358]

I would also echo the sentiment to just be able to make a pypy 3.10 environment. While I don't publish libraries, I do work in a place where we make heavy use of pypy, where it underpins our use-cases which are pure python without heavy use of the c bindings. Yes those use cases do exist 😄. We only process a couple of billion requests a year in our pypy services.

While I appreciate the fact that there is some more checks going on in conda-forge across packages, just having the option to set up a pypy 3.10 environment would be massive. I think having pypy available via conda-forge should not be dependent on other libraries having support for it. Like Lisandro points out, if library X wants to or does not want to support pypy should be on them and it definitely should not block pypy from providing updated versions through conda-forge.

Reading your other comments I hear the frustration about the lack of pypy uptake, but I think it is there, just hidden. For some insight, our python based services can run either in cpython or pypy, just a flag in the build config. Our build systems then create a fresh python environment, install dependencies, etc.

For cpython, we do something like conda create -n build python=[some version] as a way to get a clean environment.

For pypy we literally go to pypy.org, grab the tar, shove it on an internal file system and then pull that tar into the build container, unzip to create a new environment then do all the same stuff as for cpython.

In both cases, we are caching artifacts local to us to speed up builds, so while you might see a single download from pypy.org for us once per release, we are building services using pypy hundreds if not thousands of times a month.