search

PKU-EPIC / DexGraspNet

147 stars 17 forks source link

How to download Third Parties? #13

Closed zoctipus closed 1 week ago

zoctipus commented 4 months ago

| +-- ManifoldPlus | +-- TorchSDF

I am having difficulty figuring out how to download above third parties.

For the CoCAD, I could simply do pip install CoCAD

For the ManifoldPlus, after building it is a executable that is been downloaded, I am not sure how this is useful in python

For the TorchSDF, I could not find where is the repository.

Could you please provide the guide for running the main.py and code in script? Thank you

wrc042 commented 3 months ago

We provides some README files in subdirectory: DexGraspNet/asset_process and DexGraspNet/grasp_generation. They include detailed instructions for dependencies.

For CoACD and ManifoldPlus, they are tools writen in c++. We recommand to install it with cmake, and provide python scritpts to excute them in asset_process.

For TorchSDF, you can install it with:

cd DexGraspNet/thirdparty
git clone https://github.com/wrc042/TorchSDF.git
cd TorchSDF
git checkout 0.1.0
bash install.sh

Hope this can help you!

zoctipus commented 3 months ago

I followed the guide and installed all correct dependencies as (https://github.com/PKU-EPIC/DexGraspNet/tree/main/grasp_generation) described

however, when running bash install.sh for torchsdf

I got the following error:

(dexgraspnet) octipus@octipus:~/Projects/DexGraspNet/thirdparty/TorchSDF$ bash install.sh
setup.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources import parse_version
INFO - running develop
/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/command/develop.py:39: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
INFO - running egg_info
INFO - creating torchsdf.egg-info
INFO - writing torchsdf.egg-info/PKG-INFO
INFO - writing dependency_links to torchsdf.egg-info/dependency_links.txt
INFO - writing top-level names to torchsdf.egg-info/top_level.txt
INFO - writing manifest file 'torchsdf.egg-info/SOURCES.txt'
INFO - reading manifest file 'torchsdf.egg-info/SOURCES.txt'
INFO - adding license file 'LICENSE'
INFO - writing manifest file 'torchsdf.egg-info/SOURCES.txt'
INFO - running build_ext
/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/utils/cpp_extension.py:782: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.3). Most likely this shouldn't be a problem.
  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
INFO - building 'torchsdf._C' extension
INFO - creating /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build
INFO - creating /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37
INFO - creating /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf
INFO - creating /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc
Emitting ninja build file /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/3] c++ -MMD -MF /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/unbatched_triangle_distance.o.d -pthread -B /home/octipus/anaconda3/envs/dexgraspnet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/TH -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/THC -I/home/octipus/anaconda3/envs/dexgraspnet/include/python3.7m -c -c /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/csrc/unbatched_triangle_distance.cpp -o /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/unbatched_triangle_distance.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
[2/3] /usr/bin/nvcc  -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/TH -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/THC -I/home/octipus/anaconda3/envs/dexgraspnet/include/python3.7m -c -c /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/csrc/[unbatched_triangle_distance_cuda.cu](http://unbatched_triangle_distance_cuda.cu/) -o /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/unbatched_triangle_distance_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14
FAILED: /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/unbatched_triangle_distance_cuda.o
/usr/bin/nvcc  -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/TH -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/THC -I/home/octipus/anaconda3/envs/dexgraspnet/include/python3.7m -c -c /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/csrc/[unbatched_triangle_distance_cuda.cu](http://unbatched_triangle_distance_cuda.cu/) -o /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/unbatched_triangle_distance_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
[3/3] c++ -MMD -MF /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/bindings.o.d -pthread -B /home/octipus/anaconda3/envs/dexgraspnet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/TH -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/THC -I/home/octipus/anaconda3/envs/dexgraspnet/include/python3.7m -c -c /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/csrc/bindings.cpp -o /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/bindings.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1723, in _run_ninja_build
    env=env)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "setup.py", line 178, in <module>
    'build_ext': BuildExtension.with_options(no_python_abi_suffix=True)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/__init__.py", line 103, in setup
    return distutils.core.setup(**attrs)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/dist.py", line 963, in run_command
    super().run_command(command)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/command/develop.py", line 33, in run
    self.install_for_development()
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/command/develop.py", line 108, in install_for_development
    self.run_command('build_ext')
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/dist.py", line 963, in run_command
    super().run_command(command)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 88, in run
    _build_ext.run(self)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 735, in build_extensions
    build_ext.build_extensions(self)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 249, in build_extension
    _build_ext.build_extension(self, ext)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 555, in build_extension
    depends=ext.depends,
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 565, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1404, in _write_ninja_file_and_compile_objects
    error_prefix='Error compiling objects for extension')
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1733, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/__init__.py", line 1, in <module>
    from torchsdf.sdf import *
  File "/home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/sdf.py", line 2, in <module>
    from torchsdf import _C
ImportError: cannot import name '_C' from 'torchsdf' (/home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/__init__.py)

Do you know what may gone wrong here?

wrc042 commented 3 months ago

I am not sure about this error. It appears that the module is unable to load the C++ library _C.so. I suggest checking whether the compilation of cpp and cuda files was successful and generated _C.so? If so, I think python should find it as an importable module.

zoctipus commented 3 months ago

yes, I agree, but it appears that the issue arises from not able to build

a causing build error that I found may be below, when running bash install.sh : the build fails when subprocess calling ninja -v

[3/3] c++ -MMD -MF /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/bindings.o.d -pthread -B /home/octipus/anaconda3/envs/dexgraspnet/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -DTHRUST_IGNORE_CUB_VERSION_CHECK -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/TH -I/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/include/THC -I/home/octipus/anaconda3/envs/dexgraspnet/include/python3.7m -c -c /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/torchsdf/csrc/bindings.cpp -o /home/octipus/Projects/DexGraspNet/thirdparty/TorchSDF/build/temp.linux-x86_64-cpython-37/torchsdf/csrc/bindings.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command-line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1723, in _run_ninja_build
    env=env)
  File "/home/octipus/anaconda3/envs/dexgraspnet/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

is the command '['ninja', '-v']' correct? it seems like some argument is missing because when I run $ ninja -v in terminal directly it gives error of ninja: error: loading 'build.ninja': No such file or directory

wrc042 commented 3 months ago

I feel sorry that I have no idea. The error seems related to the setup tools, but I haven't encountered this on my machines:(

zoctipus commented 3 months ago

Don't worry! After some help with my collaborator, we built a docker on a nvidia cuda image, and was able to run the code. Thank you for being attentive to all my questions so far!