search

meidachen / STPLS3D

🔥 Synthetic and real-world 2d/3d dataset for semantic and instance segmentation (BMVC 2022 Oral)
236 stars 20 forks source link

Evaluation script for the competition of instance segmentation #15

Closed ywyue closed 2 years ago

ywyue commented 2 years ago

Hi, thanks again for the great work! For instance segmentation, you set up the HAIS baseline and provide the evaluation script to calculate AP. However, this evaluation assumes that we separate the ground truth and predication into individual blocks. For the competition, the evaluation server accepts the submission of a single .predict file for the whole point cloud (e.g. area 26). I wonder could you please also provide the evaluation script used on the evaluation server? Since the output of my current method is the prediction for the whole point cloud instead of individual blocks, I think the evaluation script used on your evaluation server would make the evaluation easier. Thanks in advance!

meidachen commented 2 years ago

Hi @ywyue,

The evaluation script is almost the same as the one from HAIS, basically, the whole point cloud will still be split into 50*50m blocks for evaluation. The script is uploaded to the HAIS folder.

ywyue commented 2 years ago

Thanks for the quick response!

YellowPuppy commented 2 years ago

Hi @ywyue,

The evaluation script is almost the same as the one from HAIS, basically, the whole point cloud will still be split into 50*50m blocks for evaluation. The script is uploaded to the HAIS folder.

@ywyue @meidachen How to use the script STPLS3DInstanceSegmentationChallenge_Codalab_Evaluate.py? Thank you!