ContinuumIO / anaconda-issues

Anaconda issue tracking
646 stars 220 forks source link

Boost regex crashes in C extension #8393

Open ehsantn opened 6 years ago

ehsantn commented 6 years ago

I'm trying to use boost::regex in a Python CPP extension module. However, it crashes with a segfault. Using std::regex works fine.

Actual Behavior

Segfault, here is gdb backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x00007fffee3b4c99 in std::__detail::_List_node_base::_M_hook (this=0x555556817900, 
    __position=0x7fffecbba860 <boost::object_cache<boost::re_detail_106501::cpp_regex_traits_base<char>, boost::re_detail_106501::cpp_regex_traits_implementation<char> >::do_get(boost::re_detail_106501::cpp_regex_traits_base<char> const&, unsigned long)::s_data>) at /home/msarahan/miniconda2/conda-bld/compilers_linux-64_1507259624353/work/.build/src/gcc-7.2.0/libstdc++-v3/src/c++98/list.cc:132
132 /home/msarahan/miniconda2/conda-bld/compilers_linux-64_1507259624353/work/.build/src/gcc-7.2.0/libstdc++-v3/src/c++98/list.cc: No such file or directory.

Expected Behavior

It should work like std::regex does.

Steps to Reproduce

Create a C extension function in a python package which creates/uses a regex object. Compile the package and run code that calls the C extension function.

Anaconda or Miniconda version:

Anaconda 5.01.

Operating System:

Ubuntu 16.04

conda info
Current conda install:

               platform : linux-64
          conda version : 4.3.30
       conda is private : False
      conda-env version : 4.3.30
    conda-build version : 3.0.30
         python version : 3.6.3.final.0
       requests version : 2.18.4
       root environment : /home/etotoni/pse-hpc/python/anaconda3_5.01  (writable)
    default environment : /home/etotoni/pse-hpc/python/anaconda3_5.01/envs/PA
       envs directories : /home/etotoni/pse-hpc/python/anaconda3_5.01/envs
                          /home/etotoni/.conda/envs
          package cache : /home/etotoni/pse-hpc/python/anaconda3_5.01/pkgs
                          /home/etotoni/.conda/pkgs
           channel URLs : https://conda.anaconda.org/ehsantn/linux-64
                          https://conda.anaconda.org/ehsantn/noarch
                          https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/intel/linux-64
                          https://conda.anaconda.org/intel/noarch
            config file : /home/etotoni/.condarc
             netrc file : None
           offline mode : False
             user-agent : conda/4.3.30 requests/2.18.4 CPython/3.6.3 Linux/4.4.0-112-generic debian/stretch/sid glibc/2.23    
                UID:GID : 1001:1001
conda list --show-channel-urls
# packages in environment at /home/etotoni/pse-hpc/python/anaconda3_5.01/envs/PA:
#
alabaster                 0.7.10           py36h306e16b_0    defaults
arrow-cpp                 0.8.0                    py36_2    conda-forge
babel                     2.5.3                    py36_0    defaults
binutils_impl_linux-64    2.28.1               h04c84fa_2    defaults
binutils_linux-64         7.2.0               hc67e822_15    defaults
boost                     1.65.1                   py36_4    defaults
bzip2                     1.0.6                h6d464ef_2    defaults
ca-certificates           2017.08.26           h1d4fec5_0    defaults
cairo                     1.14.10              hdf128ce_6    defaults
certifi                   2017.11.5        py36hf29ccca_0    defaults
coverage                  4.4.2                    py36_0    conda-forge
coveralls                 1.2.0                    py36_0    conda-forge
cycler                    0.10.0           py36h93f1223_0    defaults
cython                    0.26.1           py36h21c49d0_0    defaults
daal                      2018.0.1.20171012               1    intel
daal-devel                2018.0.1.20171012               1    intel
daal-include              2018.0.1.20171012               1    intel
dbus                      1.10.22              h3b5a359_0    defaults
docopt                    0.6.2                    py36_0    conda-forge
docutils                  0.14             py36hb0f60f5_0    defaults
expat                     2.2.5                he0dffb1_0    defaults
ffmpeg                    3.4                  h7264315_0    defaults
fontconfig                2.12.4               h88586e7_1    defaults
freetype                  2.8                  hab7d2ae_1    defaults
funcsigs                  1.0.2                     <pip>
gcc_impl_linux-64         7.2.0                hc5ce805_2    defaults
gcc_linux-64              7.2.0               haf1f6fa_15    defaults
gfortran_impl_linux-64    7.2.0                hb3c8cce_2    defaults
gfortran_linux-64         7.2.0               hd729bf5_15    defaults
glib                      2.53.6               h5d9569c_2    defaults
graphite2                 1.3.10               hc526e54_0    defaults
grpcio                    1.8.2                     <pip>
gst-plugins-base          1.12.2               he3457e5_0    defaults
gstreamer                 1.12.2               h4f93127_0    defaults
gxx_impl_linux-64         7.2.0                hd3faf3d_2    defaults
gxx_linux-64              7.2.0               h2e50c1c_15    defaults
h5py                      2.7.1            py36hb0addd7_2    ehsantn
h5py                      2.7.1.post0               <pip>
harfbuzz                  1.5.0                h2545bd6_0    defaults
hdf5-parallel             1.8.19               hf01a4b5_0    ehsantn
hpat-parquet-reader       0.1                  h3b0098f_2    ehsantn
icu                       58.2                 h9c2bf20_1    defaults
imagesize                 0.7.1            py36h52d8127_0    defaults
intel-openmp              2018.0.0             hc7b2577_8    defaults
jasper                    1.900.1              hd497a04_4    defaults
jinja2                    2.10             py36ha16c418_0    defaults
jpeg                      9b                   h024ee3a_2    defaults
libboost                  1.65.1               habcd387_4    defaults
libedit                   3.1                  heed3624_0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc                    7.2.0                h69d50b8_2    defaults
libgcc-ng                 7.2.0                h7cc24e2_2    defaults
libgfortran               3.0.0                         1    defaults
libgfortran-ng            7.2.0                h9f7466a_2    defaults
libopus                   1.2.1                hb9ed12e_0    defaults
libpng                    1.6.32               hbd3595f_4    defaults
libprotobuf               3.4.1                h5b8497f_0    defaults
libstdcxx-ng              7.2.0                h7a57d05_2    defaults
libtiff                   4.0.9                h28f6b97_0    defaults
libvpx                    1.6.1                h888fd40_0    defaults
libxcb                    1.12                 hcd93eb1_4    defaults
libxml2                   2.9.4                h2e8b1d7_6    defaults
llvmlite                  0.22.0dev0       py36hef76395_7    numba
lz4                       0.10.1                   py36_0    defaults
lz4-c                     1.8.0                h2d24295_0    defaults
markupsafe                1.0              py36hd9260cd_1    defaults
matplotlib                2.1.1            py36ha26af80_0    defaults
mkl                       2018.0.1             h19d6760_4    defaults
mpich                     3.2                           5    conda-forge
ncurses                   6.0                  h9df7e31_2    defaults
numpy                     1.13.3           py36ha12f23b_0    defaults
opencv                    3.3.1            py36h6cbbc71_1    defaults
openssl                   1.0.2m               h26d622b_1    defaults
pandas                    0.21.0           py36h78bd809_1    defaults
parquet-cpp               1.4.0.pre                     0    conda-forge
pcre                      8.41                 hc27e229_1    defaults
pip                       9.0.1            py36h6c6f9ce_4    defaults
pixman                    0.34.0               hceecf20_3    defaults
pkg-config                0.29.2               he821fa1_6    defaults
pkgconfig                 1.2.2            py36h08d3f6d_1    defaults
protobuf                  3.4.1            py36h306e679_0    defaults
protobuf                  3.5.1                     <pip>
py-boost                  1.65.1           py36hf484d3e_4    defaults
pyarrow                   0.8.0                    py36_0    conda-forge
pygments                  2.2.0            py36h0d3125c_0    defaults
pyparsing                 2.2.0            py36hee85983_1    defaults
pyqt                      5.6.0            py36h0386399_5    defaults
python                    3.6.3                h6c0c0dc_5    defaults
python-dateutil           2.6.1            py36h88d3b88_1    defaults
pytz                      2017.3           py36h63b9c63_0    defaults
pyyaml                    3.12                     py36_1    conda-forge
qt                        5.6.2               h974d657_12    defaults
readline                  7.0                  ha6073c6_4    defaults
requests                  2.13.0                   py36_0    conda-forge
scipy                     1.0.0            py36hbf646e7_0    defaults
setuptools                36.5.0           py36he42e2e1_0    defaults
sip                       4.18.1           py36h51ed4ed_2    defaults
six                       1.11.0           py36h372c433_1    defaults
snowballstemmer           1.2.1            py36h6febd40_0    defaults
sphinx                    1.6.6                    py36_0    defaults
sphinx_bootstrap_theme    0.5.3                    py36_0    conda-forge
sphinxcontrib             1.0              py36h6d0f590_1    defaults
sphinxcontrib-websupport  1.0.1            py36hb5cb234_1    defaults
sqlite                    3.20.1               hb898158_2    defaults
tbb                       2018_20170919                 0    conda-forge
tk                        8.6.7                hc745277_3    defaults
tornado                   4.5.3                    py36_0    defaults
typing                    3.6.2            py36h7da032a_0    defaults
wheel                     0.30.0           py36hfd4bba0_1    defaults
xz                        5.2.3                h55aa19d_2    defaults
yaml                      0.1.6                         0    conda-forge
zlib                      1.2.11               ha838bed_2    defaults
gmarkall commented 4 months ago

I've just run into this issue - will post an update if I determine what the problem is (I think it's something locale-related).

gmarkall commented 3 months ago

Create a C extension function in a python package which creates/uses a regex object. Compile the package and run code that calls the C extension function.

Unfortunately it seems not to quite be as simple as this. For example, the following module:

#define PY_SSIZE_T_CLEAN
#include "Python.h"

#include <boost/regex.hpp>

static PyObject*
match_regex(PyObject *self, PyObject *args) {
  const char *arg;
  if (!PyArg_ParseTuple(args, "s", &arg))
    return nullptr;

  boost::regex r(":.*");

  if (boost::regex_match(arg, r))
    Py_RETURN_TRUE;
  else
    Py_RETURN_FALSE;
}

static PyMethodDef testmethods[] = {
    {"match_regex", match_regex, METH_VARARGS, "Match regex"},
    {NULL, NULL, 0, NULL}
};

static PyModuleDef testmodule = {
  PyModuleDef_HEAD_INIT,
  "test",
  "Test module with a boost::regex",
  -1,
  testmethods,
};

PyMODINIT_FUNC
PyInit_test(void)
{
  return PyModule_Create(&testmodule);
}

Built using a setup.py:

from setuptools import setup, Extension

module = Extension(
    'test',
    sources=['test.cpp'],
    libraries=['boost_regex'],
)

setup(
    name="test",
    ext_modules=[module]
)

run with

python setup.py build_ext --inplace

and then used like, e.g.:

$ python -c "import test; print(test.match_regex(':x'))"
True

$ python -c "import test; print(test.match_regex('a:'))"
False

seems to work. So there's some other factor at play in the crash, likely induced by some additional library / package.

gmarkall commented 3 months ago

I think this is encountered when the object using Boost has been built using a GCC (and therefore libstdc++) newer than the one in the Anaconda environment, and when the Anaconda environment's libstdc++ is encountered first during the linker's search for the libstdc++ library - ensuring that the latest libstdc++ is on the search path first resolves the issue.

I think this is not really an Anaconda issue per se, and this might as well be closed (if anyone's minding this issue / repo).