`include_directories` is ignored in conda environment

[Midnighter]

Issue:

I'm trying to build a Python package that uses cmake, pybind11, GLPK, and SUNDIALS. Within the linked repository you can see that I can build a ubuntu based Docker image just fine. However, when I try to do the same within a conda environment compilation fails because the include directories are not added correctly to the compilation commands.

Steps to reproduce:

I have created an isolated conda environment with:

FROM continuumio/miniconda3:4.7.12

ENV PYTHONUNBUFFERED=1

ARG UID=1000
ARG GID=1000
ARG USER_=dunamis

RUN groupadd --system --gid "${GID}" "${USER_}" \
    && useradd --system --create-home --shell /bin/bash --home-dir "/home/${USER_}" \
        --uid "${UID}" --gid "${USER_}" "${USER_}"

COPY .condarc /root/

COPY .condarc "/home/${USER_}/"

COPY environment.yml /opt/

RUN set -eux \
    && conda env create --file /opt/environment.yml --prefix "/home/${USER_}/.dfba" \
    && conda clean --all --force-pkgs-dirs --yes \
    && chown --recursive "${USER_}" "/home/${USER_}"

WORKDIR "/home/${USER_}"

USER dunamis

RUN set -eux \
    && conda init bash

which uses .condarc

auto_activate_base: false
channel_priority: strict
channels:
  - conda-forge
  - bioconda
  - defaults

and environment.yml

name: dfba
dependencies:
- binutils-meta
- cmake
- compilers
- cxx-compiler
- git
- glpk=4.65
- make
- pkg-config
- pybind11
- python=3.7
- sundials=5.0.0

Once you run the Docker image, activate the environment with conda activate /home/dunamis/.dfba, then download the package curl -O https://gitlab.com/davidtourigny/dynamic-fba/-/archive/master/dynamic-fba-master.tar.gz. Extract the source and check compiler arguments:

tar -xzf dynamic-fba-master.tar.gz
cmake dynamic-fba-master
make -n

You should see that only -I/home/dunamis/.dfba/include/python3.7m is in the compiler arguments. If you edit the CMakeLists.txt file and change the include_directories command to add_compile_options("-I${GLPK_INCLUDE_DIR}") this is added and the package compiles correctly.

Do you have any idea what could cause these different behaviours?

Environment (conda list):

``` $ conda list # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 0_gnu conda-forge binutils-meta 1.0.4 0 conda-forge binutils_impl_linux-64 2.33.1 he1b5a44_7 conda-forge binutils_linux-64 2.33.1 h9595d00_16 conda-forge bzip2 1.0.8 h516909a_2 conda-forge c-compiler 1.0.4 h516909a_0 conda-forge ca-certificates 2019.11.28 hecc5488_0 conda-forge certifi 2019.11.28 py37_0 conda-forge cmake 3.16.2 h28c56e5_0 conda-forge compilers 1.0.4 0 conda-forge curl 7.65.3 hf8cf82a_0 conda-forge cxx-compiler 1.0.4 hc9558a2_0 conda-forge expat 2.2.5 he1b5a44_1004 conda-forge fortran-compiler 1.0.4 he991be0_0 conda-forge gcc_impl_linux-64 7.3.0 hd420e75_4 conda-forge gcc_linux-64 7.3.0 h553295d_16 conda-forge gettext 0.19.8.1 hc5be6a0_1002 conda-forge gfortran_impl_linux-64 7.3.0 hdf63c60_4 conda-forge gfortran_linux-64 7.3.0 h553295d_16 conda-forge git 2.24.0 pl526hce37bd2_1 conda-forge glpk 4.65 he80fd80_1002 conda-forge gmp 6.1.2 hf484d3e_1000 conda-forge gxx_impl_linux-64 7.3.0 hdf63c60_4 conda-forge gxx_linux-64 7.3.0 h553295d_16 conda-forge krb5 1.16.4 h2fd8d38_0 conda-forge ld_impl_linux-64 2.33.1 h53a641e_7 conda-forge libblas 3.8.0 14_openblas conda-forge libcblas 3.8.0 14_openblas conda-forge libcurl 7.65.3 hda55be3_0 conda-forge libedit 3.1.20170329 hf8c457e_1001 conda-forge libffi 3.2.1 he1b5a44_1006 conda-forge libgcc-ng 9.2.0 h24d8f2e_2 conda-forge libgfortran-ng 7.3.0 hdf63c60_4 conda-forge libgomp 9.2.0 h24d8f2e_2 conda-forge libiconv 1.15 h516909a_1005 conda-forge liblapack 3.8.0 14_openblas conda-forge libopenblas 0.3.7 h5ec1e0e_6 conda-forge libssh2 1.8.2 h22169c7_2 conda-forge libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge libuv 1.34.0 h516909a_0 conda-forge make 4.2.1 h14c3975_2004 conda-forge metis 5.1.0 he1b5a44_1005 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge openssl 1.1.1d h516909a_0 conda-forge pcre 8.43 he1b5a44_0 conda-forge perl 5.26.2 h516909a_1006 conda-forge pip 19.3.1 py37_0 conda-forge pkg-config 0.29.2 h516909a_1006 conda-forge pybind11 2.4.3 py37hc9558a2_0 conda-forge python 3.7.6 h357f687_2 conda-forge readline 8.0 hf8c457e_0 conda-forge rhash 1.3.6 h14c3975_1001 conda-forge setuptools 44.0.0 py37_0 conda-forge sqlite 3.30.1 hcee41ef_0 conda-forge suitesparse 5.6.0 h717dc36_0 conda-forge sundials 5.0.0 h0186668_3 conda-forge tbb 2019.9 hc9558a2_1 conda-forge tk 8.6.10 hed695b0_0 conda-forge wheel 0.33.6 py37_0 conda-forge xz 5.2.4 h14c3975_1001 conda-forge zlib 1.2.11 h516909a_1006 conda-forge ```

Details about conda and system ( conda info ):

``` $ conda info active environment : /home/dunamis/.dfba active env location : /home/dunamis/.dfba shell level : 1 user config file : /home/dunamis/.condarc populated config files : /home/dunamis/.condarc conda version : 4.7.12 conda-build version : not installed python version : 3.7.4.final.0 virtual packages : base environment : /opt/conda (read only) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://conda.anaconda.org/bioconda/linux-64 https://conda.anaconda.org/bioconda/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /opt/conda/pkgs /home/dunamis/.conda/pkgs envs directories : /home/dunamis/.conda/envs /opt/conda/envs platform : linux-64 user-agent : conda/4.7.12 requests/2.22.0 CPython/3.7.4 Linux/5.3.0-7625-generic debian/10 glibc/2.28 UID:GID : 1000:1000 netrc file : None offline mode : False ```

[leej3]

I encountered this issue too. I think it is the cmake binary itself that is causing the problem. I can compile/install as I expect for a range of versions of cmake but for a conda distributed cmake binaries I can trigger this failure for header include directories. This error occurs inside and outside of the conda build environment. More specifically I observe this for cmake version 3.14.0 on the default channel and for the conda-forge channel 3.13.4 >= broken < 3.17.0.

Currently fixes include:

• set the cmake version according to your version/channel needs

It looks like you can't set the channel within the yaml file but you can build with something along the lines of conda build -c conda-forge .

EDIT: I do not know the intended behavior so calling it "broken" may be a misnomer

[traversaro]

I think that this issue is similar to https://github.com/conda-forge/cmake-feedstock/issues/129, but related to implicit include directories. If I print CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES in the same system described in that issue, I get:

-- CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES: /home/traversaro/miniconda3/envs/cmake-tests/include;/home/traversaro/miniconda3/envs/cmake-tests/x86_64-conda-linux-gnu/include/c++/9.3.0;/home/traversaro/miniconda3/envs/cmake-tests/x86_64-conda-linux-gnu/include/c++/9.3.0/x86_64-conda-linux-gnu;/home/traversaro/miniconda3/envs/cmake-tests/x86_64-conda-linux-gnu/include/c++/9.3.0/backward;/home/traversaro/miniconda3/envs/cmake-tests/lib/gcc/x86_64-conda-linux-gnu/9.3.0/include;/home/traversaro/miniconda3/envs/cmake-tests/lib/gcc/x86_64-conda-linux-gnu/9.3.0/include-fixed;/home/traversaro/miniconda3/envs/cmake-tests/x86_64-conda-linux-gnu/include;/home/traversaro/miniconda3/envs/cmake-tests/x86_64-conda-linux-gnu/sysroot/usr/include

But if I try to check the actual default include directories of the compiler, I get:

#include "..." search starts here:
#include <...> search starts here:
 /home/traversaro/miniconda3/envs/cmake-tests/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0
 /home/traversaro/miniconda3/envs/cmake-tests/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/x86_64-conda-linux-gnu
 /home/traversaro/miniconda3/envs/cmake-tests/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/backward
 /home/traversaro/miniconda3/envs/cmake-tests/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/include
 /home/traversaro/miniconda3/envs/cmake-tests/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/include-fixed
 /home/traversaro/miniconda3/envs/cmake-tests/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include
 /home/traversaro/miniconda3/envs/cmake-tests/bin/../x86_64-conda-linux-gnu/sysroot/usr/include
End of search list.

This means that any call to include_directories("/home/traversaro/miniconda3/envs/cmake-tests/include") would be ignored, even if /home/traversaro/miniconda3/envs/cmake-tests/include is not considered by default by the compiler.

[jakirkham]

Is this still an issue?

[ntorresalberto]

Is this still an issue?

Yes, I think I've just ran into the same problem in a personal project and googling a bit (I didn't test the example though).

[jakirkham]

@ntorresalberto do you have a minimal reproducer that you can share? 🙂

[ntorresalberto]

@ntorresalberto do you have a minimal reproducer that you can share? slightly_smiling_face

I apologize for the delay and for the bad info. You were very sharp to request a minimal reproducer because now I don't think I can. I'm not sure what went wrong in my first tries before, but if I ever find out I'll report back.

[willian-m]

I encountered this issue too. I think it is the cmake binary itself that is causing the problem. I can compile/install as I expect for a range of versions of cmake but for a conda distributed cmake binaries I can trigger this failure for header include directories. This error occurs inside and outside of the conda build environment. More specifically I observe this for cmake version 3.14.0 on the default channel and for the conda-forge channel 3.13.4 >= broken < 3.17.0.

Currently fixes include:

• set the cmake version according to your version/channel needs

It looks like you can't set the channel within the yaml file but you can build with something along the lines of conda build -c conda-forge .

EDIT: I do not know the intended behavior so calling it "broken" may be a misnomer

I found a similar behavior with cmake 3.20 in a conda environment. On my makefile I had

find_package(Boost REQUIRED COMPONENTS program_options)
include_directories(${Boost_INCLUDE_DIRS})

and I was getting a "boost/program_options.hpp" not found even after a conda install boost. After I downgraded to v3.12, my project compiled normally.

[lionjo]

I had the same error as @willian-m using cmake 2.24 in a conda environment: Boost wasn't found despite doing

include_directories(${Boost_INCLUDE_DIRS})
find_package(Boost REQUIRED COMPONENTS program_options)

A simple

unset CXX_FLAGS

solved it for me. Apparantly cmake checks if the include directory exists in $CXX_FLAGS and if yes, does not include it.

[isuruf]

Going to close this issue. Please open a new one with the required information and a minimal reproducer.

[Midnighter]

You have a reproducible example in my original post, no?

[isuruf]

@Midnighter, would you be able to give a minimal reproducer instead of the complex reproducer you have here?

[Midnighter]

I'll try. Plenty of other things on my plate now so it will take me a bit.

[WilliamTambellini]

Still same issue here with cmake 3.24.1. The missing path is indeed in CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES : 08:45:52 -- CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES: libraries/python-3.8.0/include/python3.8 which is NOT used/added when calling g++ so compilation error. tbc.