search

EnyaHermite / SPH3D-GCN

Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds
MIT License
169 stars 32 forks source link

The experiment results of the source code did not reach the results declared by the paper. #12

Open LinXiLuo opened 4 years ago

LinXiLuo commented 4 years ago

Hi @hlei-ziyan, thanks for sharing your excellent work. However, I found the experiment results on modelnet40 were a little lower than the paper shown. In my experiment, the avg class acc of 10000 points is 88.3, and the instance acc is 90.2. In your paper, the avg class acc of 10000 points is 89.3, and the instance acc is 92.1.

my experiments platform: Python 3.5 Tensorflow 1.12.0 Cuda 9.2
Ubuntu 16.04 GPU is NVIDIA 1080Ti dataset: modelnet40_normal_resampled ( with point 10000) net config and parameters: same as the config and parameters suggested in the paper.

If there is any problem, please let me know.

EnyaHermite commented 3 years ago

Did you apply test augmentation? You may try more epochs, or the trained model we shared.

LinXiLuo commented 3 years ago

Did you apply test augmentation? You may try more epochs, or the trained model we shared.

@hlei-ziyan Thanks for your reply. Yes, I applied test augmentation and trained 250 epochs. I tried the published model, but the result is still close to the previous result.

The evaluate log is as follows,

Namespace(batch_size=32, dataset='modelnet40_normal_resampled', gpu=0, log_dir='log/trained-model-master', model='SPH3D_modelnet', model_name='model.ckpt-119', num_votes=12) eval accuracy: 0.901540 eval avg class acc: 0.870506 airplane: 1.000 bathtub: 0.860 bed: 0.990 bench: 0.800 bookshelf: 0.950 bottle: 0.980 bowl: 0.950 car: 1.000 chair: 0.980 cone: 0.950 cup: 0.700 curtain: 0.950 desk: 0.895 door: 0.850 dresser: 0.884 flower_pot: 0.250 glass_box: 0.910 guitar: 1.000 keyboard: 1.000 lamp: 0.850 laptop: 1.000 mantel: 0.910 monitor: 0.940 night_stand: 0.651 person: 0.800 piano: 0.900 plant: 0.760 radio: 0.850 range_hood: 0.940 sink: 0.850 sofa: 0.950 stairs: 0.850 stool: 0.700 table: 0.850 tent: 0.950 toilet: 1.000 tv_stand: 0.930 vase: 0.790 wardrobe: 0.650 xbox: 0.800 testing one batch require 2069.35 milliseconds

LinXiLuo commented 3 years ago

@hlei-ziyan If you can release the code corresponding to the paper, I will be very grateful.