Cross compile instead of emulating

[isuruf]

Checklist

• [ ] Used a personal fork of the feedstock to propose changes
• [ ] Bumped the build number (if the version is unchanged)
• [ ] Reset the build number to 0 (if the version changed)
• [ ] Re-rendered with the latest conda-smithy (Use the phrase code>@<space/conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
• [ ] Ensured the license file is being packaged.

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict. Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[h-vetinari]

What do we gain by cross-compilation (except speed)? We're still not running the test suite for the artefacts we're building, so while I'm happy with the speed improvement, the PPC builds might continue being broken? Or is there a test-farm like for osx-arm that tests the cross-compile builds?

[isuruf]

We're still not running the test suite for the artefacts we're building, so while I'm happy with the speed improvement, the PPC builds might continue being broken?

We are running the tests using QEMU. Building is cross-compiling, so it is fast

[h-vetinari]

We are running the tests using QEMU. Building is cross-compiling, so it is fast

Ah, that is very cool!

However, this still runs into #186, unfortunately (check the PPC runs, the test suite fails during collection, but this isn't propagated through to the output of scipy.test due to https://github.com/pytest-dev/pytest/issues/8986).

[h-vetinari]

@mattip

It seems setuptools is expecting something from distutils that pypy might not be providing? Weird though that this only affects aarch64?

    File "/home/conda/feedstock_root/[...]/venv/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 621, in _safe_data_files
      return build_py.get_data_files_without_manifest()
    File "/home/conda/feedstock_root/[...]/lib-python/3/distutils/cmd.py", line 103, in __getattr__
      raise AttributeError(attr)
  AttributeError: get_data_files_without_manifest
  Preparing metadata (pyproject.toml): finished with status 'error'

[mattip]

I recently fixed this for NumPy in numpy/numpy#20299 by pinning the setuptools version. A fix is already in setuptools in pypa/setuptools#2855, and I think it should be available soon as 58.5.3.

[h-vetinari]

Yeah, I just saw this occur on another feedstock, and then found https://github.com/conda-forge/setuptools-feedstock/issues/208, as well as the issue you linked. The new setuptools version has already been packaged by conda-forge, only needs to get through the CDN now.

Sorry for the false alarm!

[mattip]

On the one hand, projects with complicated workflows really should pin setuptools. But on the other-hand, if everyone uses old versions, setuptools will never discover it has broken someone's workflow. As suggested in the issue, if setuptools could somehow test its HEAD against more projects that would give an indication, but then it would slow down their CI significantly.

[h-vetinari]

The pypy build on aarch hangs forever in fft/tests/test_multithreading.py::test_mixed_threads_processes, source here.

Any idea what could cause this @mattip?

(as before, the only build that was rerun is the last failing one; theoretically it could be that - like with the setuptools regression - other builds would be affected as well, but just haven't been been rerun due to previous success)

[mariusvniekerk]

The pypy build on aarch hangs forever in fft/tests/test_multithreading.py::test_mixed_threads_processes

We could potentially just skip that test for the pypy on aarch64? We're already skipping some of the test suite so its probably not that terrible?

[mattip]

Skipping on pypy + aarch64 sounds reasonable. This is inside a qemu environment, right? It might be confusing the PyPy JIT. Although then I would expect other things to break as well. A first step may be to debug pypy + aarch64 in the pypy3.6-feedstock.

[jakirkham]

It could also be oversubscription of the resources provided given it is using multiple threads (and/or processes)

[h-vetinari]

I will take care of getting CI green (if need be by skipping tests) in the other two open PRs - first the version bump and then 3.10.

[jakirkham]

Thanks all! 😀