CUDA extension compilation failed on MacOS 10.13.6 with Pytorch 1.5.0

[TomHeaven]

When submitting a bug report, please include the following information (where relevant):

• OS: Mac SO 10.13.6
• PyTorch version: 1.5.0
• How you installed PyTorch (conda, pip, source): source
• Python version: 3.7.0
• CUDA/cuDNN version: 10.0, 7.4
• GPU models and configuration: Nvidia TitanV
• GCC version (if compiling from source): AppleClang++ 9.0

In addition, including the following information will also be very helpful for us to diagnose the problem:

• A script to reproduce the bug. Please try to provide as minimal of a test case as possible.

  cd cuda
  python3 setup.py develop

• Error messages and/or stack traces of the bug

  [2/2] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/site-packages/torch/include -I/usr/local/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/site-packages/torch/include/TH -I/usr/local/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c -c /Users/tomheaven/Downloads/extension-cpp-master/cuda/lltm_cuda_kernel.cu -o /Users/tomheaven/Downloads/extension-cpp-master/cuda/build/temp.macosx-10.13-x86_64-3.7/lltm_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=lltm_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=sm_70 -std=c++14
  FAILED: /Users/tomheaven/Downloads/extension-cpp-master/cuda/build/temp.macosx-10.13-x86_64-3.7/lltm_cuda_kernel.o 
  /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.7/site-packages/torch/include -I/usr/local/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.7/site-packages/torch/include/TH -I/usr/local/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c -c /Users/tomheaven/Downloads/extension-cpp-master/cuda/lltm_cuda_kernel.cu -o /Users/tomheaven/Downloads/extension-cpp-master/cuda/build/temp.macosx-10.13-x86_64-3.7/lltm_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=lltm_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=sm_70 -std=c++14
  /usr/local/lib/python3.7/site-packages/torch/include/c10/util/variant.h(2241): error: parameter pack "Ts" was referenced but not expanded

The error seems to be from pytorch's header file variant.h(2241).

template <typename... Ts>
  class variant {
    static_assert(0 < sizeof...(Ts),
                  "variant must consist of at least one alternative.");

    static_assert(lib::all<!std::is_array<Ts>::value...>::value,
                  "variant can not have an array type as an alternative.");

    static_assert(lib::all<!std::is_reference<Ts>::value...>::value,
                  "variant can not have a reference type as an alternative.");

    static_assert(lib::all<!std::is_void<Ts>::value...>::value,
                  "variant can not have a void type as an alternative.");

    public:
    template <
        typename Front = lib::type_pack_element_t<0, Ts...>, // Line 2241. The error is in this line.
        lib::enable_if_t<std::is_default_constructible<Front>::value, int> = 0>
    inline constexpr variant() noexcept(
        std::is_nothrow_default_constructible<Front>::value)
        : impl_(in_place_index_t<0>{}) {}

[zhanghang1989]

Any followup or solutions?

[hoffbrinkle]

The crux of it is here:

https://github.com/pytorch/pytorch/blob/5acc27c00a131c3e8ce4dad6e06222bb40f30dc7/c10/util/variant.h#L281

clang supports a __type_pack_element builtin. So the code will be generated using that. However, some downstream Nvidia tools, not so much. I think it's cudafe++ that can't handle it. In any case, comment out that check or whatever you need to do to make MPARK_TYPE_PACK_ELEMENT be undefined. The code will then be generated which will use generic unpacking from a pack.