Strange compilation error in cutorch

vadimkantorov commented 7 years ago

I tried to install LuaTorch from master branch (to run some old code) and got this:

[ 36%] Building NVCC (Device) object lib/THC/CMakeFiles/THC.dir/generated/THC_generated_THCTensorMathPointwiseChar.cu.o
.../torch/extra/cutorch/lib/THC/generic/THCTensorMath.cu(10): internal error: assertion failed: traverse_expr: bad expr kind (/dvs/p4/build/sw/rel/gpu_drv/r361/r361_00/drivers/compiler/edg/EDG_4.10/src/il_walk.c, line 3429)

vadimkantorov commented 7 years ago

and then got this:

/usr/local/cuda-8.0/include/thrust/system/cuda/detail/bulk/detail/cuda_launcher/cuda_launcher.hpp: In static member function ‘static thrust::system::cuda::detail::bulk_::detail::cuda_launcher_base<block_size_, ExecutionGroup, Closure>::size_type thrust::system::cuda::detail::bulk_::detail::cuda_launcher_base<block_size_, ExecutionGroup, Closure>::max_active_blocks_per_multiprocessor(const thrust::system::cuda::detail::bulk_::detail::device_properties_t&, const thrust::system::cuda::detail::bulk_::detail::function_attributes_t&, thrust::system::cuda::detail::bulk_::detail::cuda_launcher_base<block_size_, ExecutionGroup, Closure>::size_type, thrust::system::cuda::detail::bulk_::detail::cuda_launcher_base<block_size_, ExecutionGroup, Closure>::size_type) [with unsigned int block_size_ = 0u; ExecutionGroup = thrust::system::cuda::detail::bulk_::parallel_group<thrust::system::cuda::detail::bulk_::concurrent_group<> >; Closure = thrust::system::cuda::detail::bulk_::detail::closure<thrust::system::cuda::detail::for_each_n_detail::for_each_kernel, thrust::tuple<thrust::system::cuda::detail::bulk_::detail::cursor<0u>, thrust::pointer<thrust::tuple<__half, long int, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type>, thrust::detail::execute_with_allocator<THCThrustAllocator, thrust::system::cuda::detail::execute_on_stream_base>, thrust::use_default, thrust::use_default>, thrust::detail::wrapped_function<thrust::detail::allocator_traits_detail::gozer, void>, long int, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type> >; thrust::system::cuda::detail::bulk_::detail::cuda_launcher_base<block_size_, ExecutionGroup, Closure>::size_type = int]’:
/usr/local/cuda-8.0/include/thrust/system/cuda/detail/bulk/detail/cuda_launcher/cuda_launcher.hpp:80:18: internal compiler error: Segmentation fault
   static size_type max_active_blocks_per_multiprocessor(const device_properties_t &props,
                  ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
CMake Error at THC_generated_THCTensorMode.cu.o.cmake:266 (message):
  Error generating file
  .../torch/extra/cutorch/build/lib/THC/CMakeFiles/THC.dir//./THC_generated_THCTensorMode.cu.o

vadimkantorov commented 7 years ago

Doing a few more tries strangely fixed the build, but the internal crashes within nvcc AND gcc look very bizarre :)

torch / torch7

Strange compilation error in cutorch #1065