akloss
commented
3 years ago This is probably way too late, but if anyone has similar a similar issue: I think the file cuda_kernel_helper.h has been renamed to gpu_kernel_helper.h at some point, so this has to be changed in all of the import statements for newer tensorflow versions
I am trying to build and run the FlexConv as explained on GitHub, I am pretty stuck already for a few days. I have Cuda 10.1 and now I have installed python 3.6 and TensorFlow 1.14 which is built with Cuda 10. I have set my .bashrc like this:
CUDA
export OPT_PATH="/graphics/opt/opt_Ubuntu18.04" export CUDA_INSTALL_PATH=${OPT_PATH}/cuda/toolkit_10.1/cuda export CUDA_PATH=$CUDA_INSTALL_PATH export MW_NVCC_PATH=${CUDA_INSTALL_PATH}/bin/ export CUDA_LIB_PATH=${CUDA_INSTALL_PATH}/lib64 export CUDA_INC_PATH=${CUDA_INSTALL_PATH}/include export LD_LIBRARY_PATH=${CUDA_LIB_PATH}:$LD_LIBRARY_PATH export PATH=${CUDA_INSTALL_PATH}/bin:$PATH export PATH=${CUDA_INSTALL_PATH}/nvvm/bin/:$PATH
CUB
export CUB_INC=${OPT_PATH}/cub/cub-1.8.0
CUPTI
export CUPTI_LIB_PATH=${OPT_PATH}/cuda/toolkit_10.1/cuda/extras/CUPTI/lib64 export LD_LIBRARY_PATH=${CUPTI_LIB_PATH}:$LD_LIBRARY_PATH
CUDNN
export CUDNN_INC_PATH=${OPT_PATH}/cuda/cudnn/7.6.5_for_10.1/cuda/include export CUDNN_LIB_PATH=${OPT_PATH}/cuda/cudnn/7.6.5_for_10.1/cuda/lib64 export LD_LIBRARY_PATH=${CUDNN_LIB_PATH}:$LD_LIBRARY_PATH
CUPTI
export CUPTI_LIB_PATH=${OPT_PATH}/cuda/toolkit_10.1/cuda/extras/CUPTI/lib64 export LD_LIBRARY_PATH=${CUPIT_LIB_PATH}:$LD_LIBRARY_PATH export OPT_PATH=/graphics/opt/opt_Ubuntu18.04 export CUDA_INSTALL_PATH=${OPT_PATH}/cuda/toolkit_10.1/cuda export CUDA_PATH=$CUDA_INSTALL_PATH export MW_NVCC_PATH=${CUDA_INSTALL_PATH}/bin/
I also have change Findtensorflow.cmake and allow for version 1.14. After calling cmake I am getting errors like:
-- Reuse cached information from TensorFlow 1.14.0 -- [WARNING] The TensorFlow version has a bug (see #22766). We disable asserts using -DNDEBUG=True -- No TensorFlow-CC-LIBRARY detected -- No TensorFlow source repository detected -- CUDA_INCLUDE_DIRS: /graphics/opt/opt_Ubuntu18.04/cuda/toolkit_10.1/cuda/include -- CUDA_SAMPLE_INC: /graphics/opt/opt_Ubuntu18.04/cuda/toolkit_10.0/cuda/samples/common/inc -- Use Cuda-CUB from /graphics/opt/opt_Ubuntu18.04/cub/cub-1.8.0 -- will build custom TensorFlow operation "flex_conv" (CPU+GPU) -- will build custom TensorFlow operation "flex_deconv" (CPU+GPU) -- will build custom TensorFlow operation "flex_pool" (CPU+GPU) -- will build custom TensorFlow operation "knn_bruteforce" (CPU+GPU) -- Configuring done -- Generating done -- Build files have been written to: /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops [ 16%] Building NVCC (Device) object CMakeFiles/flex_conv_op_cu.dir/kernels/flex_conv_op_cu_generated_flex_conv_kernel_gpu.cu.cc.o [ 16%] Building NVCC (Device) object CMakeFiles/flex_pool_op_cu.dir/kernels/flex_pool_op_cu_generated_flex_pool_kernel_gpu.cu.cc.o [ 16%] Building NVCC (Device) object CMakeFiles/flex_deconv_op_cu.dir/kernels/flex_deconv_op_cu_generated_flex_deconv_kernel_gpu.cu.cc.o [ 16%] Building NVCC (Device) object CMakeFiles/knn_bruteforce_op_cu.dir/kernels/knn_bruteforce_op_cu_generated_knn_bruteforce_kernel_gpu.cu.cc.o /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/kernels/flex_deconv_kernel_gpu.cu.cc:24:10: fatal error: tensorflow/core/util/cuda_kernel_helper.h: No such file or directory
include "tensorflow/core/util/cuda_kernel_helper.h"
/home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/kernels/flex_pool_kernel_gpu.cu.cc:25:10: fatal error: tensorflow/core/util/cuda_kernel_helper.h: No such file or directory
include "tensorflow/core/util/cuda_kernel_helper.h"
compilation terminated. compilation terminated. /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/kernels/knn_bruteforce_kernel_gpu.cu.cc:34:10: fatal error: tensorflow/core/util/cuda_kernel_helper.h: No such file or directory
include "tensorflow/core/util/cuda_kernel_helper.h"
compilation terminated. /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/kernels/flex_conv_kernel_gpu.cu.cc:25:10: fatal error: tensorflow/core/util/cuda_kernel_helper.h: No such file or directory
include "tensorflow/core/util/cuda_kernel_helper.h"
compilation terminated. CMake Error at flex_deconv_op_cu_generated_flex_deconv_kernel_gpu.cu.cc.o.cmake:219 (message): Error generating /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/CMakeFiles/flex_deconv_op_cu.dir/kernels/./flex_deconv_op_cu_generated_flex_deconv_kernel_gpu.cu.cc.o CMake Error at flex_conv_op_cu_generated_flex_conv_kernel_gpu.cu.cc.o.cmake:219 (message): Error generating /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/CMakeFiles/flex_conv_op_cu.dir/kernels/./flex_conv_op_cu_generated_flex_conv_kernel_gpu.cu.cc.o CMake Error at knn_bruteforce_op_cu_generated_knn_bruteforce_kernel_gpu.cu.cc.o.cmake:219 (message): Error generating /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/CMakeFiles/knn_bruteforce_op_cu.dir/kernels/./knn_bruteforce_op_cu_generated_knn_bruteforce_kernel_gpu.cu.cc.o CMake Error at flex_pool_op_cu_generated_flex_pool_kernel_gpu.cu.cc.o.cmake:219 (message): Error generating /home/zakeri/Documents/Codes/MyCodes/FlexConvPytorch_20052020/Flex-Convolution-master/user_ops/CMakeFiles/flex_pool_op_cu.dir/kernels/./flex_pool_op_cu_generated_flex_pool_kernel_gpu.cu.cc.o CMakeFiles/flex_conv_op_cu.dir/build.make:63: recipe for target 'CMakeFiles/flex_conv_op_cu.dir/kernels/flex_conv_op_cu_generated_flex_conv_kernel_gpu.cu.cc.o' failed make[2]: [CMakeFiles/flex_conv_op_cu.dir/kernels/flex_conv_op_cu_generated_flex_conv_kernel_gpu.cu.cc.o] Error 1 CMakeFiles/flex_deconv_op_cu.dir/build.make:63: recipe for target 'CMakeFiles/flex_deconv_op_cu.dir/kernels/flex_deconv_op_cu_generated_flex_deconv_kernel_gpu.cu.cc.o' failed make[2]: [CMakeFiles/flex_deconv_op_cu.dir/kernels/flex_deconv_op_cu_generated_flex_deconv_kernel_gpu.cu.cc.o] Error 1 CMakeFiles/knn_bruteforce_op_cu.dir/build.make:63: recipe for target 'CMakeFiles/knn_bruteforce_op_cu.dir/kernels/knn_bruteforce_op_cu_generated_knn_bruteforce_kernel_gpu.cu.cc.o' failed make[2]: [CMakeFiles/knn_bruteforce_op_cu.dir/kernels/knn_bruteforce_op_cu_generated_knn_bruteforce_kernel_gpu.cu.cc.o] Error 1 CMakeFiles/flex_pool_op_cu.dir/build.make:63: recipe for target 'CMakeFiles/flex_pool_op_cu.dir/kernels/flex_pool_op_cu_generated_flex_pool_kernel_gpu.cu.cc.o' failed make[2]: [CMakeFiles/flex_pool_op_cu.dir/kernels/flex_pool_op_cu_generated_flex_pool_kernel_gpu.cu.cc.o] Error 1 CMakeFiles/Makefile2:141: recipe for target 'CMakeFiles/flex_conv_op_cu.dir/all' failed make[1]: [CMakeFiles/flex_conv_op_cu.dir/all] Error 2 make[1]: Waiting for unfinished jobs.... CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/knn_bruteforce_op_cu.dir/all' failed make[1]: [CMakeFiles/knn_bruteforce_op_cu.dir/all] Error 2 CMakeFiles/Makefile2:178: recipe for target 'CMakeFiles/flex_deconv_op_cu.dir/all' failed make[1]: [CMakeFiles/flex_deconv_op_cu.dir/all] Error 2 CMakeFiles/Makefile2:326: recipe for target 'CMakeFiles/flex_pool_op_cu.dir/all' failed make[1]: [CMakeFiles/flex_pool_op_cu.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: [all] Error 2
So I added two lines to FindTensorFlow.cmake:
set(TensorFlow_INCLUDE_DIR) & set(TensorFlow_LIBRARY)
And calling CMake again, I am getting this:
-- Reuse cached information from TensorFlow 1.14.0 -- [WARNING] The TensorFlow version has a bug (see #22766). We disable asserts using -DNDEBUG=True -- No TensorFlow-CC-LIBRARY detected -- No TensorFlow source repository detected -- CUDA_INCLUDE_DIRS: /graphics/opt/opt_Ubuntu18.04/cuda/toolkit_10.1/cuda/include -- CUDA_SAMPLE_INC: /samples/common/inc CMake Error at CMakeLists.txt:26 (message): requires 'export CUB_INC=/path/to/cub
I went through all the closed issues on GitHub and the instructions, I do not know what can I do further to make this built and run. Would you please help in this regard?