search

conda-forge / boost-feedstock

A conda-smithy repository for boost.
BSD 3-Clause "New" or "Revised" License
15 stars 41 forks source link

Boost.Python and segmentation faults with numpy 1.21 #127

Closed bkpoon closed 9 months ago

bkpoon commented 2 years ago

Issue:

Due to NEP 42 being accepted as being the new datatype system for numpy 1.21, C++ extensions wrapped with Boost.Python will get segmentation faults. There is an issue created upstream by @jawsc-alan (https://github.com/boostorg/python/issues/376). @dwpaley did the work to find the commit where the behavior changed. It is also possible to reproduce the behavior in numpy 1.20 by building with the NPY_USE_NEW_CASTINGIMPL environment variable.

Most likely, Boost.Python will need to be updated to handle the new datatype system.

PR to enable testing NEP 42 (https://github.com/numpy/numpy/pull/17401) PR to set NEP 42 as the default (https://github.com/numpy/numpy/pull/18676)


Environment (conda list):

``` $ conda list ```


Details about conda and system ( conda info ):

``` $ conda info ```
dwpaley commented 2 years ago

A fix in numpy is described in this issue: https://github.com/boostorg/python/issues/376 and I will open a numpy PR. The problem involved dereferencing a null pointer when checking convertibility of types (like boost_python ones) that haven't implemented the new numpy casting implementation.

bkpoon commented 2 years ago

So numpy>=1.21.5 has a temporary fix for this. Would it make sense to make a PR to https://github.com/conda-forge/conda-forge-repodata-patches-feedstock so that boost does not use numpy >=1.21,<1.21.5 if both dependencies are specified?

jakirkham commented 2 years ago

That could make sense. Though we will want to include that constraint in the package here as well

cc @conda-forge/core (in cases others have thoughts here)

bkpoon commented 2 years ago

Oops, numpy is always specified. I can make one or both PRs. Would run_constrained be the appropriate way of modifying the pin_compatible entry for numpy in this recipe? It looks like the additional arguments for pin_compatible are for including versions, not excluding versions.

jakirkham commented 1 year ago

Are people still running into this issue?

dwpaley commented 1 year ago

The problem is invisible with numpy >= 1.21.5 due to np PR 20616. Nominally that is a temporary fix that may be removed at some point. At the time I had made some type of reproducer in a container that could demonstrate this problem; I'm happy to try to pull it back together if you are interested in revisiting this.

jakirkham commented 1 year ago

Fair. Well I think the answer then is to fix in Boost.Python (as Sebastian put it in the issue). We can of course backport a patch once that makes its way upstream, but that would be the best use of time (if you have any and no worries if not)

bkpoon commented 1 year ago

We do nightly builds and tests of our conda package so we'll know if the temporary fix is no longer available pretty quickly. The Boost.Python and NumPy developers are aware of the issue, so I guess we can close this issue for now. Or should we keep it open for tracking?

h-vetinari commented 9 months ago

Closing this issue as obsolete. If there are still problems with numpy + libboost-python, please let us know!