search

conda-forge / ctng-compilers-feedstock

A conda-smithy repository for ctng-compilers.
BSD 3-Clause "New" or "Revised" License
12 stars 28 forks source link

[Bug] `gxx` 11.2.0 is not compatible with `nvidia::cuda-nvcc` #103

Closed XuehaiPan closed 1 year ago

XuehaiPan commented 2 years ago

Solution to issue cannot be found in the documentation.

Issue

I'm trying to include CXX compilers and NVCC compilers in my environment file to build C extensions for Python.

Steps to reproduce the issue:

CMakeLists.txt:

cmake_minimum_required(VERSION 3.4)
project(nvcc-gxx-test LANGUAGES CXX CUDA)

find_package(CUDA REQUIRED)

environment.yaml:

name: nvcc-gxx11-test

channels:
  - defaults
  - nvidia/label/cuda-11.5.0
  - nvidia
  - conda-forge

dependencies:
  - cmake >= 3.4
  - make
  - cxx-compiler
  - gxx = 11
  - nvidia/label/cuda-11.5.0::cuda-nvcc
  - nvidia/label/cuda-11.5.0::cuda-cudart-dev

Result:

$ mamba env create --file environment.yaml --no-default-packages
anaconda/pkgs/r/linux-64                                      No change
anaconda/pkgs/r/noarch                                        No change
anaconda/pkgs/msys2/linux-64                                  No change
conda-forge/noarch                                            No change
anaconda/pkgs/main/linux-64                                   No change
anaconda/pkgs/main/noarch                                     No change
anaconda/pkgs/msys2/noarch                                    No change
conda-forge/linux-64                                          No change
pytorch/linux-64                                              No change
pytorch/noarch                                                No change
nvidia/label/cuda-11.5.0/linux-64                             No change
nvidia/linux-64                                               No change
nvidia/label/cuda-11.5.0/noarch                               No change
nvidia/noarch                                                 No change

Looking for: ["cmake[version='>=3.4']", 'make', 'cxx-compiler', 'gxx=11', 'nvidia/label/cuda-11.5.0::cuda-nvcc', 'nvidia/label/cuda-11.5.0::cuda-cudart-dev']

Transaction

  Prefix: /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test

  Updating specs:

   - cmake[version='>=3.4']
   - make
   - cxx-compiler
   - gxx=11
   - nvidia/label/cuda-11.5.0::cuda-nvcc
   - nvidia/label/cuda-11.5.0::cuda-cudart-dev

  Package                                      Version  Build        Channel                                                                Size
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  Install:
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

  + _libgcc_mutex                                  0.1  main         mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + _openmp_mutex                                  5.1  1_gnu        mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + _sysroot_linux-64_curr_repodata_hack             3  haa98f57_10  mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch               Cached
  + binutils_impl_linux-64                        2.38  h2a08ee3_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + binutils_linux-64                           2.38.0  hc2dff05_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + bzip2                                        1.0.8  h7b6447c_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + c-ares                                      1.18.1  h7f8727e_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + ca-certificates                          2022.4.26  h06a4308_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + cmake                                       3.22.1  h1fce559_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + cuda-cccl                                  11.7.58  hc415cf5_0   nvidia/linux-64                                                      Cached
  + cuda-cudart                                11.5.50  h79feb7f_0   nvidia/label/cuda-11.5.0/linux-64                                    Cached
  + cuda-cudart-dev                            11.5.50  h2215e51_0   nvidia/label/cuda-11.5.0/linux-64                                    Cached
  + cuda-nvcc                                  11.5.50  h8f81028_0   nvidia/label/cuda-11.5.0/linux-64                                    Cached
  + cxx-compiler                                 1.0.0  hf484d3e_0   mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64     Cached
  + expat                                        2.4.4  h295c915_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + gcc                                         11.2.0  h702ea55_10  mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64     Cached
  + gcc_impl_linux-64                           11.2.0  h1234567_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + gcc_linux-64                                11.2.0  h5c386dc_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + gxx                                         11.2.0  h702ea55_10  mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64     Cached
  + gxx_impl_linux-64                           11.2.0  h1234567_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + gxx_linux-64                                11.2.0  hc2dff05_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + kernel-headers_linux-64                     3.10.0  h57e8cba_10  mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch               Cached
  + krb5                                        1.19.2  hac12032_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + ld_impl_linux-64                              2.38  h1181459_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libcurl                                     7.82.0  h0b77cf5_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libedit                               3.1.20210910  h7f8727e_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libev                                         4.33  h7f8727e_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libgcc-devel_linux-64                       11.2.0  h1234567_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libgcc-ng                                   11.2.0  h1234567_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libgomp                                     11.2.0  h1234567_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libnghttp2                                  1.46.0  hce63b2e_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libssh2                                     1.10.0  h8f2d780_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libstdcxx-devel_linux-64                    11.2.0  h1234567_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libstdcxx-ng                                11.2.0  h1234567_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + libuv                                       1.40.0  h7b6447c_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + lz4-c                                        1.9.3  h295c915_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + make                                         4.2.1  h1bed415_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + ncurses                                        6.3  h5eee18b_3   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + openssl                                     1.1.1p  h5eee18b_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + rhash                                        1.4.1  h3c74f83_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + sysroot_linux-64                              2.17  h57e8cba_10  mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch               Cached
  + xz                                           5.2.5  h7f8727e_1   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + zlib                                        1.2.12  h7f8727e_2   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached
  + zstd                                         1.5.2  ha4553b6_0   mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64             Cached

  Summary:

  Install: 44 packages

  Total download: 0 B

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Downloading and Extracting Packages
cxx-compiler-1.0.0   | 4 KB      | #################################################################################################################################################### | 100% 
gcc-11.2.0           | 24 KB     | #################################################################################################################################################### | 100% 
gxx-11.2.0           | 24 KB     | #################################################################################################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate nvcc-gxx11-test
#
# To deactivate an active environment, use
#
#     $ conda deactivate

$ mkdir build
$ cd build
$ conda activate nvcc-gxx11-test
$ cmake ..                      
-- The CXX compiler identification is GNU 11.2.0
-- The CUDA compiler identification is unknown
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - failed
-- Check for working CUDA compiler: /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/nvcc
-- Check for working CUDA compiler: /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/nvcc - broken
CMake Error at /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/share/cmake-3.22/Modules/CMakeTestCUDACompiler.cmake:56 (message):
  The CUDA compiler

    "/home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/nvcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/PanXuehai/test/build-gxx11/CMakeFiles/CMakeTmp

    Run Build Command(s):/home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/make -f Makefile cmTC_45341/fast && /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/make  -f CMakeFiles/cmTC_45341.dir/build.make CMakeFiles/cmTC_45341.dir/build
    make[1]: Entering directory '/home/PanXuehai/test/build-gxx11/CMakeFiles/CMakeTmp'
    Building CUDA object CMakeFiles/cmTC_45341.dir/main.cu.o
    /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/nvcc      -c /home/PanXuehai/test/build-gxx11/CMakeFiles/CMakeTmp/main.cu -o CMakeFiles/cmTC_45341.dir/main.cu.o
    Linking CUDA executable cmTC_45341
    /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/bin/cmake -E cmake_link_script CMakeFiles/cmTC_45341.dir/link.txt --verbose=1
    "" -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/lib -Wl,-rpath-link,/home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/lib -L/home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test/lib  CMakeFiles/cmTC_45341.dir/main.cu.o -o cmTC_45341 
    Error running link command: No such file or directory
    make[1]: *** [CMakeFiles/cmTC_45341.dir/build.make:99: cmTC_45341] Error 2
    make[1]: Leaving directory '/home/PanXuehai/test/build-gxx11/CMakeFiles/CMakeTmp'
    make: *** [Makefile:127: cmTC_45341/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)

-- Configuring incomplete, errors occurred!
See also "/home/PanXuehai/test/build-gxx11/CMakeFiles/CMakeOutput.log".
See also "/home/PanXuehai/test/build-gxx11/CMakeFiles/CMakeError.log".

Everything works fine if I change spec gxx = 11 to gxx = 10.

Installed packages

# packages in environment at /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
_sysroot_linux-64_curr_repodata_hack 3                   haa98f57_10  
binutils_impl_linux-64    2.38                 h2a08ee3_1  
binutils_linux-64         2.38.0               hc2dff05_0  
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.18.1               h7f8727e_0  
ca-certificates           2022.4.26            h06a4308_0  
cmake                     3.22.1               h1fce559_0  
cuda-cccl                 11.7.58              hc415cf5_0  
cuda-cudart               11.5.50              h79feb7f_0  
cuda-cudart-dev           11.5.50              h2215e51_0  
cuda-nvcc                 11.5.50              h8f81028_0  
cxx-compiler              1.0.0                hf484d3e_0  
expat                     2.4.4                h295c915_0  
gcc                       11.2.0              h702ea55_10  
gcc_impl_linux-64         11.2.0               h1234567_1  
gcc_linux-64              11.2.0               h5c386dc_0  
gxx                       11.2.0              h702ea55_10  
gxx_impl_linux-64         11.2.0               h1234567_1  
gxx_linux-64              11.2.0               hc2dff05_0  
kernel-headers_linux-64   3.10.0              h57e8cba_10  
krb5                      1.19.2               hac12032_0  
ld_impl_linux-64          2.38                 h1181459_1  
libcurl                   7.82.0               h0b77cf5_0  
libedit                   3.1.20210910         h7f8727e_0  
libev                     4.33                 h7f8727e_1  
libgcc-devel_linux-64     11.2.0               h1234567_1  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libnghttp2                1.46.0               hce63b2e_0  
libssh2                   1.10.0               h8f2d780_0  
libstdcxx-devel_linux-64  11.2.0               h1234567_1  
libstdcxx-ng              11.2.0               h1234567_1  
libuv                     1.40.0               h7b6447c_0  
lz4-c                     1.9.3                h295c915_1  
make                      4.2.1                h1bed415_1  
ncurses                   6.3                  h5eee18b_3  
openssl                   1.1.1p               h5eee18b_0  
rhash                     1.4.1                h3c74f83_1  
sysroot_linux-64          2.17                h57e8cba_10  
xz                        5.2.5                h7f8727e_1  
zlib                      1.2.12               h7f8727e_2  
zstd                      1.5.2                ha4553b6_0

Environment info

active environment : nvcc-gxx11-test
    active env location : /home/PanXuehai/Miniconda3/envs/nvcc-gxx11-test
            shell level : 1
       user config file : /home/PanXuehai/.condarc
 populated config files : /home/PanXuehai/.condarc
          conda version : 4.13.0
    conda-build version : not installed
         python version : 3.9.12.final.0
       virtual packages : __linux=5.10.102.1=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/PanXuehai/Miniconda3  (writable)
      conda av data dir : /home/PanXuehai/Miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/noarch
          package cache : /home/PanXuehai/Miniconda3/pkgs
                          /home/PanXuehai/.conda/pkgs
       envs directories : /home/PanXuehai/Miniconda3/envs
                          /home/PanXuehai/.conda/envs
               platform : linux-64
             user-agent : conda/4.13.0 requests/2.28.0 CPython/3.9.12 Linux/5.10.102.1-microsoft-standard-WSL2 ubuntu/20.04.3 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False