A Closer Look at Local Aggregation Operators in Point Cloud Analysis

By Ze Liu, Han Hu, Yue Cao, Zheng Zhang, Xin Tong

Updates

Oct 9, 2020: release more pytorch models for PartNet and S3DIS.
Sep 19, 2020: add shapenetpart segmentation.
July 3, 2020: initial release.

Introduction

This repo is the official implementation of "A Closer Look at Local Aggregation Operators in Point Cloud Analysis", which provides clean and the best (to-date) implementations for several representative operators including, Point MLP based (PointNet++-Like), Pseudo Grid based (KPConv-Like) and Adapt Weights (ContinuousConv-Like). It also includes a new family of local aggregation operators without learnable weights, named Position Pooling (PosPool), which is simpler than previous operators but performs similarly well or slightly better. Both PyTorch and TensorFlow implementations are given.

Three datasets are tested, including ModelNet, S3DIS and PartNet. Our implementations all achieve (or are close to) the state-of-the-art accuracy on these benchmarks by proper configurations of each operator type. In particular, one settings achieves 53.8 part category mean IoU on PartNet test set, which outperforms previous best implementations by 7.4 mIoU.

Citation

@article{liu2020closerlook3d,
  title={A Closer Look at Local Aggregation Operators in Point Cloud Analysis},
  author={Liu, Ze and Hu, Han and Cao, Yue and Zhang, Zheng and Tong, Xin},
  journal={ECCV},
  year={2020}
}

Main Results

ModelNet40

Method	Acc	Tensorflow Model	Pytorch Model
Point-wise MLP	92.8	Google / Baidu(wquw)	Google / Baidu(fj13)
Pseudo Grid	93.0	Google / Baidu(lvw4)	Google / Baidu(gmh5)
Adapt Weights	93.0	Google / Baidu(6zrg)	Google / Baidu(bbus)
PosPool	92.9	Google / Baidu(pkzd)	Google / Baidu(wuuv)
PosPool*	93.2	Google / Baidu(mjb1)	Google / Baidu(qcc6)

S3DIS

Method	mIoU	Tensorflow Model	Pytorch Model
Point-wise MLP	66.2	Google / Baidu(4mhy)	Google / Baidu(53as)
Pseudo Grid	65.9	Google / Baidu(06ta)	Google / Baidu(8skn)
Adapt Weights	66.5	Google / Baidu(7w43)	Google / Baidu(b7zv)
PosPool	66.5	Google / Baidu(gqqe)	Google / Baidu(z752)
PosPool*	66.7	Google / Baidu(qtkw)	Google / Baidu(r96f)

PartNet

Method	mIoU (val)	mIoU (test)	Tensorflow Model	Pytorch Model
Point-wise MLP	48.1	51.2	Google / Baidu(zw15)	Google / Baidu(wxff)
Pseudo Grid	50.8	53.0	Google / Baidu(0mtr)	Google / Baidu(n6b7)
Adapt Weights	50.1	53.5	Google / Baidu(551l)	Google / Baidu(pc22)
PosPool	50.0	53.4	Google / Baidu(rb4x)	Google / Baidu(3qv5)
PosPool*	50.6	53.8	Google / Baidu(2ts3)	Google / Baidu(czyq)

ShapeNetPart

Method	mIoU	msIoU	Acc	Pytorch Model
Point-wise MLP	85.7	84.1	94.5	Google / Baidu(mi2m)
Pseudo Grid	86.0	84.3	94.6	Google / Baidu(wde6)
Adapt Weights	85.9	84.5	94.6	Google / Baidu(dy1k)
PosPool	85.9	84.6	94.6	Google / Baidu(r2tr)
PosPool*	86.2	84.8	94.8	Google / Baidu(27ie)

Notes:

Overall accuracy for ModelNet40, mean IoU for S3DIS with Area-5, mean part-category IoU for PartNet are reported.
Point-wise MLP denotes PointNet++-like operators.
Pseudo Grid denotes KPConv-like operators.
Adapt Weights denotes ContinuousConv-like operators.
PosPool is a new parameter-free operator.
PosPool* denotes the sin/cos embedding variant of PosPool (see description in the paper).

Install

For tensorflow users, please refer to README.md for more detailed instructions. Our main experiments are conducted using this code base.
For pytorch users, please refer to README.md for more detailed instructions.

Acknowledgements

Our tensorflow codes borrowed a lot from KPCONV.

License

The code is released under MIT License (see LICENSE file for details).

zeliu98 / CloserLook3D

readme