conda-forge / scipy-feedstock

A conda-smithy repository for scipy.
BSD 3-Clause "New" or "Revised" License
6 stars 36 forks source link

Re-enable pypy #208

Closed h-vetinari closed 1 year ago

h-vetinari commented 2 years ago

Taken out from #206 because the builds came in late during the rc-phase and aren't yet fully fixed.

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

h-vetinari commented 2 years ago

@mattip

The error on windows is of course unchanged by https://github.com/scipy/scipy/pull/16756; interestingly, even though I didn't touch TestDIRECT::test_segmentation_fault[False] (except clarifying the name), it didn't show up anymore. However, there's now another crash on linux

=================================== FAILURES ===================================
________________________ stats/tests/test_resampling.py ________________________
[gw1] linux -- Python 3.8.13 $PREFIX/bin/python
worker 'gw1' crashed while running 'stats/tests/test_resampling.py::TestMonteCarloHypothesisTest::test_against_cramervonmises[-0.5]'

OSX looks good on the other hand!

h-vetinari commented 2 years ago

Ah, and aarch seems to segfault, but that might be because the pypy builds haven't been manually updated yet?

h-vetinari commented 1 year ago

@mattip

Finally picking up this PR again... On windows, this fails at import time:

[...]
import: 'scipy.optimize'
import: 'scipy.signal'
Traceback (most recent call last):
  File "messagestream.pyx", line 91, in scipy._lib.messagestream.MessageStream.close
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: b'C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\scipy-1p_otf2d'
Exception ignored in: 'scipy._lib.messagestream.MessageStream.__dealloc__'

On linux, all but one tests pass, but that test crashes hard:

=================================== FAILURES ===================================
________________________ stats/tests/test_hypotests.py _________________________
[gw0] linux -- Python 3.8.13 $PREFIX/bin/python
worker 'gw0' crashed while running 'stats/tests/test_hypotests.py::TestCvm_2samp::test_large_sample'

Also worth noting is that pypy38 and 39 crash in different tests

=================================== FAILURES ===================================
___________________________ stats/tests/test_fit.py ____________________________
[gw0] linux -- Python 3.9.12 $PREFIX/bin/python
worker 'gw0' crashed while running 'stats/tests/test_fit.py::TestFit::test_foldnorm'

(i.e. the corruption could have another cause that shows up randomly later)

Link to CI run

conda-forge-webservices[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

h-vetinari commented 1 year ago

OK, this is starting to look better! :)

Unix is now passing on pypy, though windows runs into a weird compilation error:

scipy\special\Faddeeva.cc(811): error C2039: '_copysign': is not a member of 'std'

It's weird because Faddeeva.cc contains all the includes that could possibly required for this (see here and here), and even more mysterious (to me) because it only happens with PyPy.

@mattip, thoughts?

mattip commented 1 year ago

The command line is

C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\cl.exe
    /c /nologo /Ox /W3 /GL /DNDEBUG /MD
    -D_USE_MATH_DEFINES
    -I%SRC_DIR%\scipy\special
    -I%PREFIX%\lib\site-packages\numpy\core\include
    -I%PREFIX%\include
    -Iscipy\_lib
    -Iscipy\_lib\boost
    -Iscipy\_build_utils\src
    -IC:\\bld\\scipy_1674792886749\\_h_env\\lib\\site-packages\\numpy\\core\\include
    -I%PREFIX%\lib\site-packages\numpy\core\include
    -Ibuild\src.win-amd64-3.8\numpy\distutils\include
    -I%PREFIX%\include
    -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\include
    -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include
    -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um
    -IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt
    -IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared
    -IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um
    -IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt
    -IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt
    -I%PREFIX%\Library\include
    -I%PREFIX%\Library\include
    /EHsc /Tpscipy\special\Faddeeva.cc 
    /Fobuild\temp.win-amd64-3.8\Release\scipy\special\Faddeeva.obj
    /std:c++14 /Zm1000

A couple of random guesses:

mattip commented 1 year ago

The line in question uses std::copysign. So maybe somewhere there is a if PYPY: #define copysign _copysign

I couldn't find any such override in numpy==1.20.3, a later numpy, cython, nor PyPy.

mattip commented 1 year ago

Whoops: got it. This recipe is using pypy3.9 v7.3.9. There was a define in that version. It was removed in 7.3.10. Can you update the dependency?

mattip commented 1 year ago

And just for traceability, here is the commit which is part of v7.3.10

h-vetinari commented 1 year ago

Can you update the dependency?

Can't do that here, it needs a PR to https://github.com/conda-forge/pypy-meta-feedstock to enable the builds from https://github.com/conda-forge/pypy3.6-feedstock/pull/99 (though I guess we're waiting for the aarch/ppc builds to be created/uploaded...?)

mattip commented 1 year ago

ahh, right. Let me see if I can move that forward.

h-vetinari commented 1 year ago

ahh, right. Let me see if I can move that forward.

I'm updating the pypy-meta PR, but cannot reasonably help with the pypy builds. :)

h-vetinari commented 1 year ago

@mattip @rgommers It seems that with the new PyPy 7.3.11, we're back on track here! 🥳