search

deephealthproject / pyecvl

Python wrapper for the ECVL.
MIT License
4 stars 2 forks source link

ImRead segfault under certain conditions #64

Open simleo opened 2 years ago

simleo commented 2 years ago

Tried pyecvl (installed from Conda) in Google Colab and found out that ImRead crashes with a segmentation fault. As shown in this notebook, This only happens in Python, while the C++ version of the program runs fine. This problem does not seem Colab-specific: this other notebook installs ecvl and pyecvl from source and ImRead works fine. One difference between the two is that in the former everything is compiled with GCC 8 (the reason is explained here), while in the latter the existing GCC 7 is used. However, Conda packages are all tested before release and no such problem was ever found: here, one difference is that they are tested on Debian 10 (same OS where they are built), whereas the Colab backend (from what I've seen so far, at least) runs Ubuntu 18.04. So this might be something subtle that's only exposed under specific circumstances.

The binding for ImRead is as follows:

  m.def("ImRead", [](const std::string& filename) -> ecvl::Image {
      ecvl::Image dst;
      if (!ecvl::ImRead(filename, dst)) {
          throw std::runtime_error("Can't read " + filename);
      }
      return dst;
      });

where m is the Python module object. Here I'm leveraging the implicit conversion between std::string and ecvl::filesystem::path for filename. The difference between GCC 7 and 8 is that ecvl::filesystem uses std::experimental::filesystem in the former case and std::filesystem in the latter. I wonder if the binding shown above can be considered faulty depending on the filesystem backend and if there's a way to make it more robust with respect to these implementation changes.

simleo commented 2 years ago

Update: tested installation from Conda on Ubuntu 18.04, no errors:

Dockerfile:

FROM ubuntu:18.04

ENV PATH="/opt/conda/bin:${PATH}"

RUN apt-get -y update && apt-get -y install wget && \
    wget -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
    bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /opt/conda && \
    conda update -y conda && \
    conda install -y -n base -c conda-forge mamba

And then:

root@59cfbabfe821:/# mamba --version
mamba 0.15.3
conda 4.10.3
root@59cfbabfe821:/# conda config --add channels dhealth
root@59cfbabfe821:/# conda config --add channels bioconda
root@59cfbabfe821:/# conda config --add channels conda-forge
root@59cfbabfe821:/# conda config --set channel_priority_strict
root@59cfbabfe821:/# mamba install python=3.7 pyecvl-cpu
root@59cfbabfe821:/# wget -O image.png https://www.w3.org/html/logo/downloads/HTML5_Badge_128.png
root@59cfbabfe821:/# cat >temp.py
import faulthandler
faulthandler.enable(all_threads=False)
import pyecvl.ecvl as ecvl
img = ecvl.ImRead("image.png")
print(img.channels_)
root@59cfbabfe821:/# python temp.py 
xyc

env dump:

root@59cfbabfe821:/# mamba env export --name root
name: root
channels:
  - conda-forge
  - bioconda
  - dhealth
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=1_llvm
  - brotlipy=0.7.0=py37h5e8e339_1003
  - bzip2=1.0.8=h7f98852_4
  - c-ares=1.18.1=h7f98852_0
  - ca-certificates=2021.10.8=ha878542_0
  - cairo=1.16.0=hcf35c78_1003
  - certifi=2021.10.8=py37h89c1867_1
  - cffi=1.14.4=py37h11fe52a_0
  - charset-normalizer=2.0.4=pyhd3eb1b0_0
  - conda=4.10.3=py37h89c1867_4
  - conda-package-handling=1.7.3=py37h5e8e339_1
  - cryptography=35.0.0=py37hf1a17b8_2
  - dbus=1.13.6=hfdff14a_1
  - dcmtk=3.6.5=h5fca97b_0
  - ecvl-cpu=0.4.2=py37h582f26e_0
  - eddl-cpu=1.0.3b0=h476a1fd_0
  - eigen=3.3.9=h4bd325d_1
  - expat=2.4.1=h9c3ff4c_0
  - ffmpeg=4.1.3=h167e202_0
  - fontconfig=2.13.1=hba837de_1005
  - freetype=2.10.4=h0708190_1
  - gdk-pixbuf=2.42.2=h19a9c64_1
  - gettext=0.19.8.1=hf34092f_1004
  - giflib=5.2.1=h36c2ea0_2
  - glib=2.66.3=h58526e2_0
  - gmp=6.2.1=h58526e2_0
  - gnutls=3.6.13=h85f3911_1
  - graphite2=1.3.13=h58526e2_1001
  - gst-plugins-base=1.14.5=h0935bb2_2
  - gstreamer=1.14.5=h36ae1b5_2
  - harfbuzz=2.4.0=h9f30f68_3
  - hdf5=1.10.5=nompi_h5b725eb_1114
  - icu=64.2=he1b5a44_1
  - idna=3.2=pyhd3eb1b0_0
  - jasper=1.900.1=h07fcdf6_1006
  - jpeg=9d=h36c2ea0_0
  - krb5=1.19.2=hcc1bbae_3
  - lame=3.100=h7f98852_1001
  - ld_impl_linux-64=2.35.1=h7274673_9
  - libarchive=3.5.1=h3f442fb_1
  - libblas=3.9.0=12_linux64_openblas
  - libcblas=3.9.0=12_linux64_openblas
  - libclang=9.0.1=default_ha53f305_2
  - libcurl=7.78.0=h0b77cf5_0
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=h516909a_1
  - libffi=3.2.1=he1b5a44_1007
  - libflac=1.3.3=h9c3ff4c_1
  - libgcc-ng=11.2.0=h1d223b6_11
  - libgfortran-ng=11.2.0=h69a702a_11
  - libgfortran5=11.2.0=h5c6108e_11
  - libglib=2.66.3=hbe7bbb4_0
  - libiconv=1.16=h516909a_0
  - libidn2=2.3.2=h7f98852_0
  - liblapack=3.9.0=12_linux64_openblas
  - liblapacke=3.9.0=12_linux64_openblas
  - libllvm9=9.0.1=default_hc23dcda_4
  - libnghttp2=1.43.0=h812cca2_1
  - libogg=1.3.4=h7f98852_1
  - libopenblas=0.3.18=pthreads_h8fe5266_0
  - libopencv=3.4.9=py37_2
  - libopus=1.3.1=h7f98852_1
  - libpng=1.6.37=h21135ba_2
  - libprotobuf=3.11.4=h8b12597_0
  - libsndfile=1.0.31=h9c3ff4c_1
  - libsolv=0.7.19=h780b84a_5
  - libssh2=1.10.0=ha56f1ee_2
  - libstdcxx-ng=11.2.0=he4da1e4_11
  - libtiff=4.1.0=hc3755c2_3
  - libunistring=0.9.10=h7f98852_0
  - libuuid=2.32.1=h7f98852_1000
  - libvorbis=1.3.7=h9c3ff4c_0
  - libwebp=1.0.2=h56121f0_5
  - libxcb=1.13=h7f98852_1004
  - libxkbcommon=0.10.0=he1b5a44_0
  - libxml2=2.9.10=hee79883_0
  - libzlib=1.2.11=h36c2ea0_1013
  - llvm-openmp=12.0.1=h4bd325d_1
  - lz4-c=1.9.3=h9c3ff4c_1
  - lzo=2.10=h516909a_1000
  - mamba=0.15.3=py37h7f483ca_0
  - ncurses=6.2=h58526e2_4
  - nettle=3.6=he412f7d_0
  - nspr=4.32=h9c3ff4c_1
  - nss=3.72=hb5efdd6_0
  - numpy=1.21.4=py37h31617e3_0
  - openh264=1.8.0=hdbcaa40_1000
  - openjpeg=2.4.0=hf7af979_0
  - openslide=3.4.1=h8137273_1
  - openssl=1.1.1l=h7f98852_0
  - pcre=8.45=h9c3ff4c_0
  - pip=21.3.1=pyhd8ed1ab_0
  - pixman=0.38.0=h516909a_1003
  - protobuf=3.11.4=py37h3340039_1
  - pthread-stubs=0.4=h36c2ea0_1001
  - py-opencv=3.4.9=py37h5ca1d4c_2
  - pycosat=0.6.3=py37h5e8e339_1009
  - pycparser=2.21=pyhd3eb1b0_0
  - pyecvl-cpu=0.12.2=py37h663d75d_0
  - pyeddl-cpu=1.1.0=py37h663d75d_0
  - pyopenssl=21.0.0=pyhd3eb1b0_1
  - pysocks=1.7.1=py37h89c1867_4
  - python=3.7.8=h8bdb77d_2_cpython
  - python_abi=3.7=2_cp37m
  - qt=5.12.5=hd8c4c69_1
  - readline=8.1=h27cfd23_0
  - reproc=14.2.3=h7f98852_0
  - reproc-cpp=14.2.3=h9c3ff4c_0
  - requests=2.26.0=pyhd3eb1b0_0
  - ruamel_yaml=0.15.80=py37h5e8e339_1006
  - setuptools=59.2.0=py37h89c1867_0
  - six=1.16.0=pyhd3eb1b0_0
  - sqlite=3.36.0=hc218d9a_0
  - tk=8.6.11=h1ccaba5_0
  - tqdm=4.62.3=pyhd3eb1b0_1
  - tzdata=2021e=hda174b7_0
  - urllib3=1.26.7=pyhd3eb1b0_0
  - wget=1.20.3=ha56f1ee_1
  - wheel=0.37.0=pyhd8ed1ab_1
  - x264=1!152.20180806=h14c3975_0
  - xorg-kbproto=1.0.7=h7f98852_1002
  - xorg-libice=1.0.10=h7f98852_0
  - xorg-libsm=1.2.3=hd9c2040_1000
  - xorg-libx11=1.7.2=h7f98852_0
  - xorg-libxau=1.0.9=h7f98852_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xorg-libxext=1.3.4=h7f98852_1
  - xorg-libxrender=0.9.10=h7f98852_1003
  - xorg-renderproto=0.11.1=h7f98852_1002
  - xorg-xextproto=7.3.0=h7f98852_1002
  - xorg-xproto=7.0.31=h7f98852_1007
  - xz=5.2.5=h7b6447c_0
  - yaml=0.2.5=h7b6447c_0
  - yaml-cpp=0.6.3=he1b5a44_4
  - zlib=1.2.11=h36c2ea0_1013
  - zstd=1.4.9=ha95c52a_0
prefix: /opt/conda