search

idiap / attention-sampling

This Python package enables the training and inference of deep learning models for very large data, such as megapixel images, using attention-sampling
Other
98 stars 18 forks source link

CMake Error at patches_generated_extract_patches.cu.o.cmake:207 #2

Closed nathanin closed 5 years ago

nathanin commented 5 years ago

I'm building your ats repository inside of a Docker container based on the latest tensorflow gpu image with this Dockerfile:

FROM tensorflow/tensorflow:latest-gpu-py3

WORKDIR /
RUN apt-get update && \
    apt-get install -y git g++ cmake

RUN git clone https://github.com/idiap/attention-sampling attention-sampling

RUN g++ --version && \
    cmake --version && \
    python --version && \
    python -c "import tensorflow; print(tensorflow.__version__)"

WORKDIR /attention-sampling
RUN pip install -e . && \
    cd ats/ops/extract_patches && \
    mkdir build && \
    cd build && \
    cmake -DCMAKE_BUILD_TYPE=Release .. && \
    make && \
    make install

WORKDIR /attention-sampling
RUN python -m unittest discover -s tests/ 

WORKDIR /

In order to build, I've set the "default-runtime":"nvidia". Some other version info:

g++ (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

cmake version 3.5.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).
Python 3.5.2
tensorflow-gpu 1.13.1

An error is encountered at cmake:

-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda (found version "10.0") 
-- Found TensorFlow: /usr/local/lib/python3.5/dist-packages/tensorflow/include  
-- Configuring done
-- Generating done
-- Build files have been written to: /attention-sampling/ats/ops/extract_patches/build
[ 33%] Building NVCC (Device) object CMakeFiles/patches.dir/patches_generated_extract_patches.cu.o
In file included from /usr/local/lib/python3.5/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/util/ConfigureVectorization.h:384:0,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/Core:22,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/Tensor:14,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/numeric_types.h:20,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:23,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23,
                 from /attention-sampling/ats/ops/extract_patches/extract_patches.cu:12:
/usr/local/cuda/include/host_defines.h:54:2: warning: #warning "host_defines.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead." [-Wcpp]
 #warning "host_defines.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead."
  ^
In file included from /usr/include/c++/5/atomic:38:0,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/lib/core/refcount.h:19,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/tensor_coding.h:21,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/resource_handle.h:19,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:24,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23,
                 from /attention-sampling/ats/ops/extract_patches/extract_patches.cu:12:
/usr/include/c++/5/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
 #error This file requires compiler and library support \
  ^
In file included from /usr/local/lib/python3.5/dist-packages/tensorflow/include/absl/base/config.h:66:0,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/absl/strings/string_view.h:31,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/lib/core/stringpiece.h:29,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/tensor_coding.h:22,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/resource_handle.h:19,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:24,
                 from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23,
                 from /attention-sampling/ats/ops/extract_patches/extract_patches.cu:12:
/usr/local/lib/python3.5/dist-packages/tensorflow/include/absl/base/policy_checks.h:77:2: error: #error "C++ versions less than C++11 are not supported."
 #error "C++ versions less than C++11 are not supported."
  ^
CMake Error at patches_generated_extract_patches.cu.o.cmake:207 (message):
  Error generating
  /attention-sampling/ats/ops/extract_patches/build/CMakeFiles/patches.dir//./patches_generated_extract_patches.cu.o

make[2]: *** [CMakeFiles/patches.dir/patches_generated_extract_patches.cu.o] Error 1
CMakeFiles/patches.dir/build.make:63: recipe for target 'CMakeFiles/patches.dir/patches_generated_extract_patches.cu.o' failed
make[1]: *** [CMakeFiles/patches.dir/all] Error 2
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/patches.dir/all' failed
make: *** [all] Error 2
Makefile:127: recipe for target 'all' failed

I am confused because I see in ats/ops/extract_patches/CMakeLists.txt you set(CMAKE_CXX_STANDARD 11), but the error is related to "C++ versions less than C++11 are not supported."? I'm not well versed in c++, is this something you've run into previously? How can I move forward?

CPU version compiles and tests successfully.

angeloskath commented 5 years ago

Hi,

Thanks for taking the time to compile and test the code. I have not come across this issue, however I do see a ton of warnings when I compile for CUDA which hints towards something not being done 100% correctly.

Since you can reproduce the error I have the following suggestions, if you want to help :). In any way thank you very much for the interest and the time.

  1. Nvidia suggests using target_compile_features(paches PUBLIC cxx_std_11) instead of set(CMAKE_CXX_STANDARD 11). Could you try keeping both?
  2. This is a bit more involved but since the library is a bit small you can try compiling directly with nvcc.

In any way if you just want to use the library you can disable cuda for compiling the extension and the wall clock time cost will not be that much. Maybe ~20%. Every convolution etc will still be happening on the GPU just the patch extraction will be happening on the CPU which is not that slow.

I will also try to reproduce the error and will post here if I manage.

Cheers, Angelos

nathanin commented 5 years ago

Hi, sorry to have just gotten back to this.

I did as you suggested and included both set(...) and target_compile_features(..). With the target_compile_features I got errors about the first argument paches, I also changed it to patches, but the error was the same: the target is not built by the project? This is out of my depth, so I just ended up commenting the if(CUDA_FOUND) block and compiled with the patch extractor CPU only.

Thanks for your help!

angeloskath commented 5 years ago

Hi,

I am closing the issue because I can't reproduce it. Feel free to reopen or open a new one if you have problems.

Cheers, Angelos