search

WXinlong / ASIS

Associatively Segmenting Instances and Semantics in Point Clouds, CVPR 2019
MIT License
255 stars 65 forks source link

reproduce results on Area5 for instance segmentation #14

Closed FrankCAN closed 5 years ago

FrankCAN commented 5 years ago

Hi, I retrained your model without any change, and test it on Area5, but the performance result is quite low compared with yours, I just wonder if there are some information I don't know.

My environment is tensorflow 1.6 and python 3.5 and my result is as below: Instance Segmentation mMUCov: 0.3599536949433257 Instance Segmentation mMWCov: 0.38854278223099514 Instance Segmentation mPrecision: 0.40252245593357444 Instance Segmentation mRecall: 0.31376601611130606

Semantic Segmentation oAcc: 0.8630127024386128 Semantic Segmentation mAcc: 0.5824638078371082 Semantic Segmentation mIoU: 0.5037184939579153

Thanks a lot for your help.

WXinlong commented 5 years ago

Hi, your results are weird. Can you help me check if the dataset including the annotations are collected and organized correctly. Or could you try again in enrironment with python2.7 and tensorflow1.3.

FrankCAN commented 5 years ago

Hi Xinlong, Thanks for your response.

I can generate .npy and h5 files successfully by following your guidance, so do it mean that the annotations should be no problem and organized correctly?

The difference is just that I only trained data/train_hdf5_file_list_woArea5.txt file instead, but I think train_hdf5_file_list_woArea1~5.txt are totally separate, have no any connection, and they only use for CV test, am I right? Should I have to train train_hdf5_file_list_woArea1 to train_hdf5_file_list_woArea5 completely?

Thanks a lot for your help.

Best Regards Frank

WXinlong commented 5 years ago

train_hdf5_file_list_woArea5.txt means the filelist of area 1,2,3,4,6.
The code was tested with Python 2.7. I suggest you use an environment with Python2.7 and TF1.3. Please refer to #12 for more help.

FrankCAN commented 5 years ago

Hi Xinlong, Thanks for your help, I got expected results, and even a little bit higher than your paper, thanks.

hu-zhiyu commented 5 years ago

It's weird that I still got unmatched Area5 instance segmentation results when using the pretrained model. I used an environment with python 2.7 and tensorflow 1.3.0 under Ubuntu 16.04. The semantic segmentation results are all good. Any idea why? Here below are the results that I got: Instance Segmentation mMUCov: 0.388768637499 Instance Segmentation mMWCov: 0.41931708045 Instance Segmentation mPrecision: 0.49950009677 Instance Segmentation mRecall: 0.357493869746

Semantic Segmentation oAcc: 0.869736688706 Semantic Segmentation mAcc: 0.609961164943 Semantic Segmentation mIoU: 0.534575881062