ICG-Net
ICG-Net: A Unified Approach for Instance Centric Grasping
About The Project
This repo contains the implementation of ICG-Net. For the Benchmark and Checkpoints, please refer to the Benchmark Repository.
Getting Started
To get a local copy up and running follow these steps:
Prerequisites
Installation
Pytorch 2.2, Cuda 12.1
To install MinkowskiEngine with Pytorch >= 2.0 and Cuda >= 12.0, you will need to install our patched version of MinkowskiEngine. ```bash sudo apt-get install libopenblas-dev export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6" # Install conda environment conda env create -f conda_cu121.yml conda activate icg_cuda121 # Install Pytorch and Dependencies pip install torch torchvision torchaudio torch_geometric pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.2.0+cu121.html # Install Patched MinkowskiEngine git clone git@github.com:renezurbruegg/MinkowskiEngine.git cd MinkowskiEngine # Link conda cuda version for minkowski compilation export CUDA_HOME=${CONDA_PREFIX}/envs/icg_cuda121 # Link cuda libraries. export LD_LIBRARY_PATH=${CONDA_PREFIX}/lib:${LD_LIBRARY_PATH} # Link cuda headers. This might not be necessary for all systems. sudo ln -s ${CONDA_PREFIX}/lib/libcudart.so.12 /usr/lib/libcudart.so python setup.py install --force_cuda --blas=openblas --blas_include_dirs=${CONDA_PREFIX}/include --cuda_home=$CUDA_HOME cd - # Install third party requirements cd icg_net/third_party/pointnet2 python setup.py install cd - # Install icg_net as pip package pip install -e . ```Pytorch 1.12 Cuda 11.3
```bash sudo apt-get install libopenblas-dev # Install conda environment export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6" conda env create -f conda_cu113.yml conda activate icg_cuda113 # Install Pytorch and Dependencies pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113 pip install torch_geometric pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-1.12.0+cu113.html # Install MinkowskiEngine git clone --recursive "https://github.com/NVIDIA/MinkowskiEngine" cd MinkowskiEngine git checkout 02fc608bea4c0549b0a7b00ca1bf15dee4a0b228 python setup.py install --force_cuda --blas=openblas # Install third party requirements cd icg_net/third_party/pointnet2 python setup.py install cd - # Install icg_net as pip package pip install -e . ```Usage
from icg_net import ICGNetModule, get_model
from icg_net.typing import ModelPredOut
# Load model
model: ICGNetModule = get_model(
"icg_benchmark/data/51--0.656/config.yaml",
device="cuda" if torch.cuda.is_available() else "cpu",
)
pc, normals = # ... load pc
out: ModelPredOut = model(
torch.from_numpy(np.asarray(o3dc.points)).float(),
normals=torch.from_numpy(np.asarray(o3dc.normals)).float(),
grasp_pts=grasp_pts,
grasp_normals=grasp_normals,
n_grasps=512,
each_object=True,
return_meshes=True,
return_scene_grasps=True,
)
print(out)
License
Distributed under the BSD-2 License. See LICENSE.txt for more information.
Citing
If you use this code in your research, please cite the following paper:
@article{zurbrugg2024icgnet,
title={ICGNet: A Unified Approach for Instance-Centric Grasping},
author={Zurbr{\"u}gg, Ren{\'e} and Liu, Yifan and Engelmann, Francis and Kumar, Suryansh and Hutter, Marco and Patil, Vaishakh and Yu, Fisher},
journal={arXiv preprint arXiv:2401.09939},
year={2024}
}