search

JonasSchult / Mask3D

Mask3D predicts accurate 3D semantic instances achieving state-of-the-art on ScanNet, ScanNet200, S3DIS and STPLS3D.
MIT License
528 stars 105 forks source link

Evaluation output file contains all values 1 #52

Closed Vishal1711 closed 1 year ago

Vishal1711 commented 1 year ago

Hello @JonasSchult,

Thanks for this Mask3D model. I trained the Mask3D model with a custom PCD dataset and followed the S3DIS dataset format. When I run inference, the .txt file is generated at eval_output/instance_evaluation_singleplytest_query_150_topk_300_dbscan_0.95_0/decoder_-1/. The output of that file is all follow:

image

Total 7 labels were used for training. Each scene contains two objects (an object of class 6 is common in scene). The length 3D point cloud data in 'Area_1_scene2_envelope.ply' is 306858 points. Of that, 226902 belongs to class 6, and 79956 points belong to class 1 in gt. I run inference for 40 scenes, and it shows the prediction for class 6 correct but either wrong or with confidence less than 0.40 for other classes. And the generated mask files have all values 1 as follows: image . . . . image

The expected value should be 1 only for points belonging to that class, and the rest values as 0. Non-zero integers indicate part of the predicted instance.

Is this due to the long length of point clouds of one label compared to the other? Or am I missing something in the code?

Hey @pamogar as you are also working on the custom dataset. Any input from your side would be helpful.

Thanks, Vishal

Vishal1711 commented 1 year ago

I kept evaluation_on_segmenet=True. Therefore, I faced this issue. After I made it False, it worked.