search

WeijingShi / Point-GNN

Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud, CVPR 2020.
MIT License
523 stars 114 forks source link

Detailed Point-GNN #50

Open wailok09 opened 3 years ago

wailok09 commented 3 years ago

Hello Weijing, I appreciate your work of Point-GNN very much.

Do you have a detailed explanation on how your coding works, for example a detailed journal article, as I don't understand the difference between run.py, train.py and eval.py? I know you have a conference article but I think that is not enough for me to gain the information. Besides, how do you visualize the KITTI point cloud dataset and your result, because I can't directly open the "large" or "old" bin file in CloudCompare? Also, how can I run your code in Windows?

I will be grateful for your assistance.

WeijingShi commented 3 years ago

Hi @wailok09,

Thanks for your interest. Train.py is for training the model and eval.py is to monitor the training progress. After the training, run.py is used for inference. The usages can be documented in README. Also, you can try python run.py --help to get descriptions of all the arguments. Unfortunately, we don't have a separate article to describe the code but we believe the code is fairly commented. This comment gives a general description.

As for visualization, run.py contains an argument -l LEVEL, --level LEVEL, visualization level, 0 to disable,1 to nonblocking, visualization, 2 to block.Default=0.

For example, you can python3 run.py checkpoints/car_auto_T3_train/ --dataset_root_dir DATASET_ROOT_DIR --output_dir DIR_TO_SAVE_RESULTS --level 2. The code generates results that's compatible to KITTI. More on the KITTI format can be found in their website.

I didn't test on Windows but all the code and packages are in python. If you can set up python in windows and install the python packages as in the README. It might be fine.

Hope it helps.

wailok09 commented 3 years ago

Hi Weijing,

I'm working on your code in my FYP thesis, how do you think your program can be improved? Is it possible to include scale invariant feature transform (SIFT) method, or you have any suggestions?

By the way, I'm grateful for your latest assistance.

WeijingShi commented 3 years ago

One of the major problems the current version has is that the running speed. Though some speedup can be achieved by code optimization, the density of the graph still needs to be better controlled to get a reasonable speed. For dense prediction, a graph feature pyramid structure is preferred.

It's a very interesting idea to combine SIFT. As the point cloud is in 3D space, we usually do not have the scale problem as in the 2D image. However, rotation-invariant point features might be helpful to point matching.

wailok09 commented 3 years ago

Hello Weijing,

Thank you for your recommendations. As for rotation-invariant features, I have found a work related to this: Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution at https://arxiv.org/pdf/1811.09361.pdf Their coding is also available at github at https://github.com/qq456cvb/PRIN Do you think his implementation of rotation invariant feature would overall improve your work?

Hope to hear you soon, thanks.