Pytorch Implementation of PointNet and PointNet++

This repo is implementation for PointNet and PointNet++ in pytorch.

Update

2021/03/27:

(1) Release pre-trained models for semantic segmentation, where PointNet++ can achieve 53.5\% mIoU.

(2) Release pre-trained models for classification and part segmentation in log/.

2021/03/20: Update codes for classification, including:

(1) Add codes for training ModelNet10 dataset. Using setting of --num_category 10.

(2) Add codes for running on CPU only. Using setting of --use_cpu.

(3) Add codes for offline data preprocessing to accelerate training. Using setting of --process_data.

(4) Add codes for training with uniform sampling. Using setting of --use_uniform_sample.

2019/11/26:

(1) Fixed some errors in previous codes and added data augmentation tricks. Now classification by only 1024 points can achieve 92.8\%!

(2) Added testing codes, including classification and segmentation, and semantic segmentation with visualization.

(3) Organized all models into ./models files for easy using.

Install

The latest codes are tested on Ubuntu 16.04, CUDA10.1, PyTorch 1.6 and Python 3.7:

conda install pytorch==1.6.0 cudatoolkit=10.1 -c pytorch

Classification (ModelNet10/40)

Data Preparation

Download alignment ModelNet here and save in data/modelnet40_normal_resampled/.

Run

You can run different modes with following codes.

If you want to use offline processing of data, you can use --process_data in the first run. You can download pre-processd data here and save it in data/modelnet40_normal_resampled/.
If you want to train on ModelNet10, you can use --num_category 10.
```
# ModelNet40
## Select different models in ./models 
```

e.g., pointnet2_ssg without normal features

python train_classification.py --model pointnet2_cls_ssg --log_dir pointnet2_cls_ssg python test_classification.py --log_dir pointnet2_cls_ssg

e.g., pointnet2_ssg with normal features

python train_classification.py --model pointnet2_cls_ssg --use_normals --log_dir pointnet2_cls_ssg_normal python test_classification.py --use_normals --log_dir pointnet2_cls_ssg_normal

e.g., pointnet2_ssg with uniform sampling

python train_classification.py --model pointnet2_cls_ssg --use_uniform_sample --log_dir pointnet2_cls_ssg_fps python test_classification.py --use_uniform_sample --log_dir pointnet2_cls_ssg_fps

ModelNet10

Similar setting like ModelNet40, just using --num_category 10

e.g., pointnet2_ssg without normal features

python train_classification.py --model pointnet2_cls_ssg --log_dir pointnet2_cls_ssg --num_category 10 python test_classification.py --log_dir pointnet2_cls_ssg --num_category 10


### Performance
| Model | Accuracy |
|--|--|
| PointNet (Official) |  89.2|
| PointNet2 (Official) | 91.9 |
| PointNet (Pytorch without normal) |  90.6|
| PointNet (Pytorch with normal) |  91.4|
| PointNet2_SSG (Pytorch without normal) |  92.2|
| PointNet2_SSG (Pytorch with normal) |  92.4|
| PointNet2_MSG (Pytorch with normal) |  **92.8**|

## Part Segmentation (ShapeNet)
### Data Preparation
Download alignment **ShapeNet** [here](https://shapenet.cs.stanford.edu/media/shapenetcore_partanno_segmentation_benchmark_v0_normal.zip)  and save in `data/shapenetcore_partanno_segmentation_benchmark_v0_normal/`.
### Run

Check model in ./models

e.g., pointnet2_msg

python train_partseg.py --model pointnet2_part_seg_msg --normal --log_dir pointnet2_part_seg_msg python test_partseg.py --normal --log_dir pointnet2_part_seg_msg

### Performance
| Model | Inctance avg IoU| Class avg IoU 
|--|--|--|
|PointNet (Official)    |83.7|80.4  
|PointNet2 (Official)|85.1  |81.9   
|PointNet (Pytorch)|    84.3    |81.1|  
|PointNet2_SSG (Pytorch)|   84.9|   81.8    
|PointNet2_MSG (Pytorch)|   **85.4**|   **82.5**    

## Semantic Segmentation (S3DIS)
### Data Preparation
Download 3D indoor parsing dataset (**S3DIS**) [here](http://buildingparser.stanford.edu/dataset.html)  and save in `data/s3dis/Stanford3dDataset_v1.2_Aligned_Version/`.

cd data_utils python collect_indoor3d_data.py

Processed data will save in `data/stanford_indoor3d/`.
### Run

Check model in ./models

e.g., pointnet2_ssg

python train_semseg.py --model pointnet2_sem_seg --test_area 5 --log_dir pointnet2_sem_seg python test_semseg.py --log_dir pointnet2_sem_seg --test_area 5 --visual

Visualization results will save in `log/sem_seg/pointnet2_sem_seg/visual/` and you can visualize these .obj file by [MeshLab](http://www.meshlab.net/).

### Performance
|Model  | Overall Acc |Class avg IoU | Checkpoint 
|--|--|--|--|
| PointNet (Pytorch) | 78.9 | 43.7| [40.7MB](log/sem_seg/pointnet_sem_seg) |
| PointNet2_ssg (Pytorch) | **83.0** | **53.5**| [11.2MB](log/sem_seg/pointnet2_sem_seg) |

## Visualization
### Using show3d_balls.py

build C++ code for visualization

cd visualizer bash build.sh

run one example

python show3d_balls.py

![](/visualizer/pic.png)
### Using MeshLab
![](/visualizer/pic2.png)

## Reference By
[halimacc/pointnet3](https://github.com/halimacc/pointnet3)<br>
[fxia22/pointnet.pytorch](https://github.com/fxia22/pointnet.pytorch)<br>
[charlesq34/PointNet](https://github.com/charlesq34/pointnet) <br>
[charlesq34/PointNet++](https://github.com/charlesq34/pointnet2)

## Citation
If you find this repo useful in your research, please consider citing it and our other works:

@article{Pytorch_Pointnet_Pointnet2, Author = {Xu Yan}, Title = {Pointnet/Pointnet++ Pytorch}, Journal = {https://github.com/yanx27/Pointnet_Pointnet2_pytorch}, Year = {2019} }

@InProceedings{yan2020pointasnl, title={PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling}, author={Yan, Xu and Zheng, Chaoda and Li, Zhen and Wang, Sheng and Cui, Shuguang}, journal={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, year={2020} }

@InProceedings{yan2021sparse, title={Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion}, author={Yan, Xu and Gao, Jiantao and Li, Jie and Zhang, Ruimao, and Li, Zhen and Huang, Rui and Cui, Shuguang}, journal={AAAI Conference on Artificial Intelligence ({AAAI})}, year={2021} }

@InProceedings{yan20222dpass, title={2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds}, author={Xu Yan and Jiantao Gao and Chaoda Zheng and Chao Zheng and Ruimao Zhang and Shuguang Cui and Zhen Li}, year={2022}, journal={ECCV} }


## Selected Projects using This Codebase
* [PointConv: Deep Convolutional Networks on 3D Point Clouds, CVPR'19](https://github.com/Young98CN/pointconv_pytorch)
* [On Isometry Robustness of Deep 3D Point Cloud Models under Adversarial Attacks, CVPR'20](https://github.com/skywalker6174/3d-isometry-robust)
* [Label-Efficient Learning on Point Clouds using Approximate Convex Decompositions, ECCV'20](https://github.com/matheusgadelha/PointCloudLearningACD)
* [PCT: Point Cloud Transformer](https://github.com/MenghaoGuo/PCT)
* [PSNet: Fast Data Structuring for Hierarchical Deep Learning on Point Cloud](https://github.com/lly007/PointStructuringNet)
* [Stratified Transformer for 3D Point Cloud Segmentation, CVPR'22](https://github.com/dvlab-research/stratified-transformer)

yanx27 / Pointnet_Pointnet2_pytorch

readme

Pytorch Implementation of PointNet and PointNet++

Update

Install

Classification (ModelNet10/40)

Data Preparation

Run

e.g., pointnet2_ssg without normal features

e.g., pointnet2_ssg with normal features

e.g., pointnet2_ssg with uniform sampling

ModelNet10

Similar setting like ModelNet40, just using --num_category 10

e.g., pointnet2_ssg without normal features

Check model in ./models

e.g., pointnet2_msg

Check model in ./models

e.g., pointnet2_ssg

build C++ code for visualization

run one example