search

charlesq34 / pointnet2

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
Other
3.06k stars 894 forks source link

Running pointnet++ on tf 1.5 #50

Open kargarisaac opened 6 years ago

kargarisaac commented 6 years ago

I use cuda 9 and can not install tf 1.2 for running pointnet++. I also disn't understand the solution in readme for tf>1.4. Can anybody help me?

RivaLei commented 6 years ago

me too, the tf on my computer is 1.5, so the compiling of tf_xxx_compile.sh always failed.

RivaLei commented 6 years ago

i can't understand the how to edit the tf_xxx_compile.sh

faisalazhar1 commented 6 years ago

I am trying to compile the tf_xxx_compile.sh in windows using pycharm with tensorflow 1.8 (with GPU). For windows, you need to first get Git to run the shell script in pycharm. Next, create a new bash script and configure the interpreter to point to sh.exe (example, D:\Program Files\Git\bin\sh.exe) Run the hello world example to make sure the shell script is working.

Now the hard part of compiling the gibberish (I say this because I am new to python+tensorflow+pycharm) code under windows. I have not been successful but managed to find some information on it so thought might be useful to other people. We need to study this link:

The code in tf_xxx_compile.sh is pointing to some paths. For my case, I used the python console to find these, e.g., : Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32 import tensorflow as tf Backend Qt5Agg is interactive backend. Turning interactive mode on. tf.sysconfig.get_lib() 'D:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow' tf.sysconfig.get_include() 'D:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\include' tf.sysconfig.get_compile_flags() ['-ID:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\include', '-D_GLIBCXX_USE_CXX11_ABI=0'] tf.sysconfig.get_link_flags() ['-LD:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow', '-ltensorflow_framework']

However, when I try to run the .sh file, I come across two errors:

Adding the lines below causes 'no module named tensorflow'. I can run .py files with importing tensorflow but the shell script can not find tensorflow. This maybe because I gave the interpreter path to sh.exe rather than tensorflow\python.exe. However, .sh does not run if you give it the latter path. TF_INC=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())') TF_LIB=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())')

Second, g++ not found.

It seems that the code is implemented in cuda with python and C++ wrappers. Given this situation, I am inclining towards implementing the sampling functions in C++/python.

eduard626 commented 5 years ago

Here is my edited version of the tf_ops compiling scripts. As you can see you need to indicate the nvcc and cuda version that you are trying to use, basically you need to install a TF version compatible with CUDA 9 and do something similar in your computer.

`#/bin/bash

/usr/local/cuda-8.0/bin/nvcc tf_grouping_g.cu -o tf_grouping_g.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

/usr/local/cuda-9.0/bin/nvcc tf_grouping_g.cu -o tf_grouping_g.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

TF1.2

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I /usr/local/lib/python2.7/dist-packages/tensorflow/include -I /usr/local/cuda-8.0/include -lcudart -L /usr/local/cuda-8.0/lib64/ -O2 -D_GLIBCXX_USE_CXX11_ABI=0

TF1.4

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I /usr/local/lib/python2.7/dist-packages/tensorflow/include -I /usr/local/cuda-8.0/include -I /usr/local/lib/python2.7/dist-packages/tensorflow/include/external/nsync/public -lcudart -L /usr/local/cuda-8.0/lib64/ -L/usr/local/lib/python2.7/dist-packages/tensorflow -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

TF1.6

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-8.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-8.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

TF1.8

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-9.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-9.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0 `

HuayueZhang commented 5 years ago

Here is my edited version of the tf_ops compiling scripts. As you can see you need to indicate the nvcc and cuda version that you are trying to use, basically you need to install a TF version compatible with CUDA 9 and do something similar in your computer.

`#/bin/bash

/usr/local/cuda-8.0/bin/nvcc tf_grouping_g.cu -o tf_grouping_g.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

/usr/local/cuda-9.0/bin/nvcc tf_grouping_g.cu -o tf_grouping_g.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

TF1.2

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I /usr/local/lib/python2.7/dist-packages/tensorflow/include -I /usr/local/cuda-8.0/include -lcudart -L /usr/local/cuda-8.0/lib64/ -O2 -D_GLIBCXX_USE_CXX11_ABI=0

TF1.4

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I /usr/local/lib/python2.7/dist-packages/tensorflow/include -I /usr/local/cuda-8.0/include -I /usr/local/lib/python2.7/dist-packages/tensorflow/include/external/nsync/public -lcudart -L /usr/local/cuda-8.0/lib64/ -L/usr/local/lib/python2.7/dist-packages/tensorflow -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

TF1.6

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-8.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-8.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

TF1.8

g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-9.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-9.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0 `

Do you know why this error occur? [[[ ’cc: error: unrecognized command line option ‘-fPIC ]]]

faisalazhar1 commented 5 years ago

Perhaps break the line after -fPIC -I g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-9.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-9.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

HuayueZhang commented 5 years ago

Perhaps break the line after -fPIC -I g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-9.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-9.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

https://github.com/charlesq34/pointnet2/issues/50#issuecomment-424339344

Perhaps break the line after -fPIC -I g++ -std=c++11 tf_grouping.cpp tf_grouping_g.cu.o -o tf_grouping_so.so -shared -fPIC -I $TF_INC -I /usr/local/cuda-9.0/include -I $TF_INC/external/nsync/public -lcudart -L /usr/local/cuda-9.0/lib64/ -L$TF_LIB -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=0

Not work... The error may occur at the first sentence: /usr/local/cuda-8.0/bin/nvcc tf_grouping_g.cu -o tf_grouping_g.cu.o -c -O2 -DGOOGLE_CUDA=1 -x cu -Xcompiler -fPIC This -fPIC is not recognized. Why...

leowang7 commented 4 years ago

Here is what I have used for TF 1.14:

One thing to point out is that the library name has been changed from libtensorflow_framework.so to libtensorflow_framework.so.1, you need to create a symbolic link here.

  1. Find tensorflow include path: tf.sysconfig.get_include()
  2. Find tensorflow library path: tf.sysconfig.get_lib())
  3. Create a symbolic link for tensorflow library. ln -s libtensorflow_framework.so.1 libtensorflow_framework.so
  4. Update the paths in shell script.

Example: g++ -std=c++11 tf_sampling.cpp tf_sampling_g.cu.o -o tf_sampling_so.so -shared -fPIC -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include -I /usr/local/cuda/include -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include/external/nsync/public -lcudart -L /usr/local/cuda/lib64/ -L/home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=1

wanyiming2017 commented 4 years ago

Here is what I have used for TF 1.14:

One thing to point out is that the library name has been changed from libtensorflow_framework.so to libtensorflow_framework.so.1, you need to create a symbolic link here.

1. Find tensorflow include path:
   `tf.sysconfig.get_include()`

2. Find tensorflow library path:
   `tf.sysconfig.get_lib())`

3. Create a symbolic link for tensorflow library.
   `ln -s libtensorflow_framework.so.1 libtensorflow_framework.so`

4. Update the paths in shell script.

Example: g++ -std=c++11 tf_sampling.cpp tf_sampling_g.cu.o -o tf_sampling_so.so -shared -fPIC -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include -I /usr/local/cuda/include -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include/external/nsync/public -lcudart -L /usr/local/cuda/lib64/ -L/home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=1

Hi, could you describe it in detailed about how create the symbolic link? I tried your method but failed

SmiletoE commented 2 years ago

Here is what I have used for TF 1.14: One thing to point out is that the library name has been changed from libtensorflow_framework.so to libtensorflow_framework.so.1, you need to create a symbolic link here.

1. Find tensorflow include path:
   `tf.sysconfig.get_include()`

2. Find tensorflow library path:
   `tf.sysconfig.get_lib())`

3. Create a symbolic link for tensorflow library.
   `ln -s libtensorflow_framework.so.1 libtensorflow_framework.so`

4. Update the paths in shell script.

Example: g++ -std=c++11 tf_sampling.cpp tf_sampling_g.cu.o -o tf_sampling_so.so -shared -fPIC -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include -I /usr/local/cuda/include -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include/external/nsync/public -lcudart -L /usr/local/cuda/lib64/ -L/home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=1

Hi, could you describe it in detailed about how create the symbolic link? I tried your method but failed

symbolic link

Here is what I have used for TF 1.14:

One thing to point out is that the library name has been changed from libtensorflow_framework.so to libtensorflow_framework.so.1, you need to create a symbolic link here.

  1. Find tensorflow include path: tf.sysconfig.get_include()
  2. Find tensorflow library path: tf.sysconfig.get_lib())
  3. Create a symbolic link for tensorflow library. ln -s libtensorflow_framework.so.1 libtensorflow_framework.so
  4. Update the paths in shell script.

Example: g++ -std=c++11 tf_sampling.cpp tf_sampling_g.cu.o -o tf_sampling_so.so -shared -fPIC -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include -I /usr/local/cuda/include -I /home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow/include/external/nsync/public -lcudart -L /usr/local/cuda/lib64/ -L/home/leo/anaconda3/envs/tf-gpu/lib/python3.7/site-packages/tensorflow -ltensorflow_framework -O2 -D_GLIBCXX_USE_CXX11_ABI=1

As a supplement, symbolic link are performed under TensorFlow's package file, you'll see the 'libtensorflow_framework.so.x' under this file. for example, the file path is '/home/yuanbo/anaconda3/envs/TF2.5/lib/python3.6/site-packages/tensorflow/', then just carry out ln -s libtensorflow_framework.so.x libtensorflow_framework.so here.