search

conda-forge / nvcc-feedstock

A conda-smithy repository for nvcc.
BSD 3-Clause "New" or "Revised" License
12 stars 23 forks source link

Internal compiler error with libstdc++ 10 #78

Closed h-vetinari closed 2 years ago

h-vetinari commented 2 years ago

Not sure where's the best place to file this, but since there are a bunch of NVIDIA folks here I thought this feedstock fits.

Encountered in https://github.com/conda-forge/fairscale-feedstock/pull/10, with the following stacktrace:

  [2/2] /usr/local/cuda/bin/nvcc  -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/work/fairscale/clib/fused_adam_cuda -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include/TH -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/include/python3.7m -c -c /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/work/fairscale/clib/fused_adam_cuda/fused_adam_cuda_kernel.cu -o /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/work/build/temp.linux-x86_64-3.7/fairscale/clib/fused_adam_cuda/fused_adam_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=fused_adam_cuda -D_GLIBCXX_USE_CXX11_ABI=1 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_86,code=sm_86 -ccbin /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_build_env/bin/x86_64-conda-linux-gnu-cc -std=c++14
  FAILED: /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/work/build/temp.linux-x86_64-3.7/fairscale/clib/fused_adam_cuda/fused_adam_cuda_kernel.o
  /usr/local/cuda/bin/nvcc  -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/work/fairscale/clib/fused_adam_cuda -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include/TH -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/include/python3.7m -c -c /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/work/fairscale/clib/fused_adam_cuda/fused_adam_cuda_kernel.cu -o /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/work/build/temp.linux-x86_64-3.7/fairscale/clib/fused_adam_cuda/fused_adam_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=fused_adam_cuda -D_GLIBCXX_USE_CXX11_ABI=1 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_86,code=sm_86 -ccbin /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_build_env/bin/x86_64-conda-linux-gnu-cc -std=c++14
  /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_build_env/x86_64-conda-linux-gnu/include/c++/10.3.0/chrono: In substitution of 'template<class _Rep, class _Period> template<class _Period2> using __is_harmonic = std::__bool_constant<(std::ratio<((_Period2::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)) * (_Period::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den))), ((_Period2::den / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::den, _Period::den)) * (_Period::num / std::chrono::duration<_Rep, _Period>::_S_gcd(_Period2::num, _Period::num)))>::den == 1)> [with _Period2 = _Period2; _Rep = _Rep; _Period = _Period]':
  /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_build_env/x86_64-conda-linux-gnu/include/c++/10.3.0/chrono:473:154:   required from here
  /home/conda/feedstock_root/build_artifacts/fairscale_1646864772118/_build_env/x86_64-conda-linux-gnu/include/c++/10.3.0/chrono:428:27: internal compiler error: Segmentation fault
    428 |  _S_gcd(intmax_t __m, intmax_t __n) noexcept

Downgrading to GCC 9 made the builds pass.

kkraus14 commented 2 years ago

Not sure where's the best place to file this, but since there are a bunch of NVIDIA folks here I thought this feedstock fits.

@h-vetinari you can submit the bug here: https://developer.nvidia.com/nvidia_bug/add (requires registration).