search

apple / ARKitScenes

This repo accompanies the research paper, ARKitScenes - A Diverse Real-World Dataset for 3D Indoor Scene Understanding Using Mobile RGB-D Data and contains the data, scripts to visualize and process assets, and training code described in our paper.
Other
662 stars 58 forks source link

Detection Evaluation Table 2 #16

Closed francisengelmann closed 2 years ago

francisengelmann commented 2 years ago

Hi all,

Thanks for the great dataset, this is very helpful!

I have a few questions regarding the evaluation shown in Table 2 in the paper: 1) Which intersection-over-union (IoU) threshold t did you use to compute mAP@t ? I assume it is either 0.5 or 0.25 as in VoteNet? 2) Is the IoU computed over axis-aligned bounding boxes, or oriented bounding boxes? 3) On which split (val or test) are these numbers reported? Since test is currently not available, what are the numbers on validation? 4) Are the per-point annotations available?

Thanks a lot!

Best, Francis

afshindn commented 2 years ago

Hi Francis,

  1. Threshold is set to 0.25
  2. We are using the oriented bounding boxes for evaluation
  3. The reported results are on validation set which is released.
  4. Currently we don't have per point annotations ready for release. We have gotten some request and will consider that in future releases.

Cheers, Afshin

PeterZheFu commented 2 years ago

Thanks all. Closing the issue for now. Feel free to re-open it~