optas / latent_3d_points

Auto-encoding & Generating 3D Point-Clouds.
Other
511 stars 111 forks source link

Cannot compile EMD makefile on Google Colab #28

Closed anguyen216 closed 3 years ago

anguyen216 commented 3 years ago

Hi,

Has anyone try to run this code on Google Colab? I'm choosing Google Colab to avoid having to install all the requirement on my local machine and to take advantage of the GPU on Google Colab.

I'm trying to make the environment as close to the one in which this code has been tested on. So far, Colab let me install CUDA 8.0.61, tensorflow 1.15.2. I'm using gcc 4.8 to compile the makefile. Python, however, is version 3.6 because that's the only thing Google Colab will support. When I tried to compile the makefile for the EMD/Chamfer losses, I get this error message. If anyone know how to work around this, I'd really appreciate it, otherwise, I'll probably have to install all the requirement on my local machine

/content/drive/My Drive/Project/latent_3d_points-master/external/structural_losses
make: Circular tf_approxmatch_g.cu <- tf_approxmatch_g.cu.o dependency dropped.
make: Circular tf_nndistance_g.cu <- tf_nndistance_g.cu.o dependency dropped.
/usr/local/cuda/bin/nvcc -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -c -o tf_nndistance_g.cu.o tf_nndistance_g.cu -I /usr/local/lib/python3.6/dist-packages/tensorflow/include -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -O2
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1414): warning: calling a constexpr __host__ function("real") from a __host__ __device__ function("abs") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1414): warning: calling a constexpr __host__ function("imag") from a __host__ __device__ function("abs") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1414): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1414): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1419): warning: calling a constexpr __host__ function("real") from a __host__ __device__ function("abs") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1419): warning: calling a constexpr __host__ function("imag") from a __host__ __device__ function("abs") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1419): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1419): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1443): warning: calling a constexpr __host__ function("real") from a __host__ __device__ function("exp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1444): warning: calling a constexpr __host__ function("imag") from a __host__ __device__ function("exp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1445): warning: calling a constexpr __host__ function("imag") from a __host__ __device__ function("exp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1443): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1444): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1445): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1446): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1451): warning: calling a constexpr __host__ function("real") from a __host__ __device__ function("exp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1452): warning: calling a constexpr __host__ function("imag") from a __host__ __device__ function("exp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1453): warning: calling a constexpr __host__ function("imag") from a __host__ __device__ function("exp") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1451): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1452): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1453): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/MathFunctions.h(1454): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(259): warning: calling a constexpr __host__ function("signbit") from a __host__ __device__ function("truncate_to_bfloat16") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(261): warning: calling a constexpr __host__ function("min") from a __host__ __device__ function("truncate_to_bfloat16") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(263): warning: calling a constexpr __host__ function("signbit") from a __host__ __device__ function("truncate_to_bfloat16") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(259): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(263): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(294): warning: calling a constexpr __host__ function("signbit") from a __host__ __device__ function("float_to_bfloat16_rtne") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(295): warning: calling a constexpr __host__ function("min") from a __host__ __device__ function("float_to_bfloat16_rtne") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(297): warning: calling a constexpr __host__ function("signbit") from a __host__ __device__ function("float_to_bfloat16_rtne") is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(294): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/Default/BFloat16.h(297): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorRandom.h(154): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorRandom.h(159): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorRandom.h(267): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorRandom.h(272): warning: calling a constexpr __host__ function from a __host__ __device__ function is not allowed. The experimental flag '--expt-relaxed-constexpr' can be used to allow this.

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/GPU/PacketMath.h(520): error: no suitable conversion function from "const Eigen::half" to "unsigned int" exists

/usr/local/lib/python3.6/dist-packages/tensorflow/include/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/GPU/PacketMath.h(521): error: class "__half2" has no member "y"

2 errors detected in the compilation of "/tmp/tmpxft_00001468_00000000-5_tf_nndistance_g.cpp4.ii".
makefile:21: recipe for target 'tf_nndistance_g.cu.o' failed
make: *** [tf_nndistance_g.cu.o] Error 2

Here's what in my makefile right now

nvcc = /usr/local/cuda/bin/nvcc
cudalib =  /usr/local/cuda/lib64
tensorflow = /usr/local/lib/python3.6/dist-packages/tensorflow/include

all: tf_approxmatch_so.so tf_approxmatch_g.cu.o tf_nndistance_so.so tf_nndistance_g.cu.o

tf_approxmatch_so.so: tf_approxmatch_g.cu.o tf_approxmatch.cpp
    g++ -std=c++11 tf_approxmatch.cpp tf_approxmatch_g.cu.o -o tf_approxmatch_so.so -shared -fPIC -I $(tensorflow) -lcudart -L $(cudalib) -O2 -D_GLIBCXX_USE_CXX11_ABI=0

tf_approxmatch_g.cu.o: tf_approxmatch_g.cu
    $(nvcc) -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -c -o tf_approxmatch_g.cu.o tf_approxmatch_g.cu -I $(tensorflow) -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -O2

tf_nndistance_so.so: tf_nndistance_g.cu.o tf_nndistance.cpp
    g++ -std=c++11 tf_nndistance.cpp tf_nndistance_g.cu.o -o tf_nndistance_so.so -shared -fPIC -I $(tensorflow) -lcudart -L $(cudalib) -O2 -D_GLIBCXX_USE_CXX11_ABI=0

tf_nndistance_g.cu.o: tf_nndistance_g.cu
    $(nvcc) -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -c -o tf_nndistance_g.cu.o tf_nndistance_g.cu -I $(tensorflow) -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -O2

clean:
    rm tf_approxmatch_so.so
    rm tf_nndistance_so.so
    rm  *.cu.o
anguyen216 commented 3 years ago

So I figured out a solution for this. At the time of writing this, Google Colab default CUDA version is 10.1 and python is 3.7; I downgrade the tensorflow on Colab to 1.15.2 so it's more compatible with CUDA 10.1 and the code written in this repo using the code below

%tensorflow_version 1.x
import sys
import tensorflow
print(tensorflow.__version__)

Also need to install gcc/g++ before I can run makefile

!apt-get install -qq gcc-5 g++-5 -y
!ln -s /usr/bin/gcc-5 
!ln -s /usr/bin/g++-5 

!sudo apt-get update
!sudo apt-get upgrade

The first three lines of the makefile can be edit as below (this is the path/directory on Google Colab; not sure if you can manually change this or that if you want to/should do that)

nvcc = /usr/local/cuda/bin/nvcc
cudalib =  /usr/local/cuda/lib64
tensorflow = /usr/local/lib/python3.6/dist-packages/tensorflow/include

then after mounting the Google Drive and change directory to makefile and cpp files location, I run make as below

%cd /content/drive/My\ Drive/path/to/makefile/
!make

and voila, it worked like a charm. Leave this here in case someone interested in doing the same thing as I'm doing right now in the future and got stuck :)