search

conda-forge / rdkit-feedstock

A conda-smithy repository for rdkit.
BSD 3-Clause "New" or "Revised" License
8 stars 22 forks source link

conda-forge RDKit segfaults with cairo=1.14 dependency #29

Closed j-wags closed 5 years ago

j-wags commented 5 years ago

Issue: While working on builds of the open force field toolkit yesterday, we found that our combination of dependencies produced a build where RDKit segfaults upon being imported (from rdkit import Chem). Subsequently running conda install rdkit would fix it. With some trial and error, we were able to make functional builds by pinning to cairo==1.16 in our meta.yaml.

I'm providing code to reproduce this problem using conda-forge. Right now I'm not able to reproduce it using the rdkit channel, but I will open an issue on the rdkit github if that changes.

Running on Mac OSX 10.14.4

Code to reproduce:

conda create --yes -n test python=3.7
conda activate test
conda install --yes -c conda-forge rdkit cairo=1.14
python -c "from rdkit import Chem"

returns

Segmentation fault: 11

The full environment is below, but I noticed that this segfaulting build resolves to

  rdkit              conda-forge/osx-64::rdkit-2018.09.1-py37hdb4e85a_1000

When we pin to cairo=1.16, we get a working build.

  rdkit              conda-forge/osx-64::rdkit-2018.09.1-py37h4e14f70_1001

I haven't been able to track down what was adding a dependency on cairo 1.14. Pinning it to 1.16 in our builds doesn't seem to have broken anything important or raised an incompatibility error.


Environment (conda list):

``` (test3) jwagner@MBP-S$ conda list WARNING: The conda.compat module is deprecated and will be removed in a future release. # packages in environment at /Users/jwagner/miniconda3/envs/test3: # # Name Version Build Channel boost 1.68.0 py37h9888f84_1001 conda-forge boost-cpp 1.68.0 h6f8c590_1000 conda-forge bzip2 1.0.6 h1de35cc_1002 conda-forge ca-certificates 2019.3.9 hecc5488_0 conda-forge cairo 1.14.12 h9247486_1006 conda-forge certifi 2019.3.9 py37_0 conda-forge clangdev 6.0.1 hba662a8_1000 conda-forge fontconfig 2.13.1 h1027ab8_1000 conda-forge freetype 2.10.0 h24853df_0 conda-forge gettext 0.19.8.1 h46ab8bc_1002 conda-forge glib 2.58.3 h2836805_1001 conda-forge icu 58.2 h0a44026_1000 conda-forge jpeg 9c h1de35cc_1001 conda-forge libblas 3.8.0 4_openblas conda-forge libcblas 3.8.0 4_openblas conda-forge libcxx 6.0.1 0 conda-forge libffi 3.2.1 h6de7cb9_1006 conda-forge libgfortran 3.0.1 0 conda-forge libiconv 1.15 h01d97ff_1005 conda-forge liblapack 3.8.0 4_openblas conda-forge libpng 1.6.36 ha441bb4_1000 conda-forge libtiff 4.0.10 h79f4b77_1001 conda-forge libxml2 2.9.8 hf14e9c8_1005 conda-forge llvm-meta 6.0.1 0 conda-forge llvmdev 6.0.1 h04f5b5a_1002 conda-forge ncurses 6.1 h0a44026_1002 conda-forge numpy 1.16.2 py37hbb3c62a_1 conda-forge olefile 0.46 py_0 conda-forge openblas 0.3.5 h436c29b_1001 conda-forge openssl 1.1.1b h01d97ff_2 conda-forge pandas 0.24.2 py37h0a44026_0 conda-forge pcre 8.41 h0a44026_1003 conda-forge pillow 6.0.0 py37h7095ceb_0 conda-forge pip 19.0.3 py37_0 conda-forge pixman 0.34.0 h1de35cc_1003 conda-forge pycairo 1.18.0 py37ha54c0a8_1000 conda-forge python 3.7.3 h0d93f26_0 conda-forge python-dateutil 2.8.0 py_0 conda-forge pytz 2018.9 py_0 conda-forge rdkit 2018.09.1 py37hdb4e85a_1000 conda-forge readline 7.0 hcfe32e1_1001 conda-forge setuptools 41.0.0 py37_0 conda-forge six 1.12.0 py37_1000 conda-forge sqlite 3.26.0 h1765d9f_1001 conda-forge tk 8.6.9 ha441bb4_1001 conda-forge wheel 0.33.1 py37_0 conda-forge xz 5.2.4 h1de35cc_1001 conda-forge zlib 1.2.11 h1de35cc_1004 conda-forge ```


Details about conda and system ( conda info ):

``` (test3) jwagner@MBP-S$ conda info active environment : test3 active env location : /Users/jwagner/miniconda3/envs/test3 shell level : 1 user config file : /Users/jwagner/.condarc populated config files : /Users/jwagner/.condarc conda version : 4.6.11 conda-build version : not installed python version : 3.7.1.final.0 base environment : /Users/jwagner/miniconda3 (writable) channel URLs : https://conda.anaconda.org/conda-forge/osx-64 https://conda.anaconda.org/conda-forge/noarch https://conda.anaconda.org/omnia-dev/osx-64 https://conda.anaconda.org/omnia-dev/noarch https://conda.anaconda.org/omnia/osx-64 https://conda.anaconda.org/omnia/noarch https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/osx-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /Users/jwagner/miniconda3/pkgs /Users/jwagner/.conda/pkgs envs directories : /Users/jwagner/miniconda3/envs /Users/jwagner/.conda/envs platform : osx-64 user-agent : conda/4.6.11 requests/2.21.0 CPython/3.7.1 Darwin/18.5.0 OSX/10.14.4 UID:GID : 501:20 netrc file : None offline mode : False ```
mcs07 commented 5 years ago

I think pull request #24 fixed this segfaulting issue on Mac. So v2018.09.1 build 1000 is the last 'broken' release and v2018.09.1 build 1001 and all later versions of RDKit are fixed. Looks like that pull request also coincided with the conda-forge ecosystem going from cairo 1.14 to 1.16, so yes, unfortunately if something is pinning cairo to the old 1.14 version, conda will also give you the old broken RDKit.

For the omnia channel, you might want to use the conda-forge-pinning package in your CI scripts to make sure the versions of the dependencies are compatible with conda-forge packages. Take a look at https://github.com/mcs07/conda-recipes/ for an example of a multi-recipe CI setup that was inspired by the conda-forge staged-recipes setup. It also uses a couple of python scripts as an alternative to the old conda-build-all tool that isn't compatible with newer conda/conda-build versions.

j-wags commented 5 years ago

Thanks for the helpful reply, @mcs07! I'll check out conda-forge-pinning.

Hopefully, we'll be moving our toolkit off omnia and into conda-forge soon. The open force field project is really looking forward to making the jump!