search

theNded / torch-ash

[PAMI 2022, CVPR 2023] ASH: Parallel Spatial Hashing for Fast Scene Reconstruction
https://dongwei.info/publication/ash-mono/
MIT License
153 stars 6 forks source link

Release of evaluation groundtruth mesh #20

Open Willyzw opened 1 year ago

Willyzw commented 1 year ago

Hi, thank you for sharing the excellent code. The code looks really clean to start with! Regarding the evaluation in the CVPR 23 paper, 4 sequences from ScanNet and another 4 from 7-Scenes datasets are evaluated for the reconstruction quality. For future comparison, I wonder if you could kindly provide the ground truth used for the evaluation, so that we can make sure the sames are compared against. Thanks a lot!

theNded commented 1 year ago

Thanks for your interest. They are all included in the example datasets mentioned in the README: https://drive.google.com/drive/folders/12E4cTIIxmShV_ENkcvzKOQunsa0TqDVQ?usp=drive_link. Maybe I should improve README to make them more salient as well!

Willyzw commented 1 year ago

Indeed, sorry I didn't notice it. The GT meshes of 7-scenes data are included in the provided link. However I couldn't find the GT meshes of scannet data.

Although the author of manhattan sdf does provide a version of GT meshes for scannet data, but the used coordinate frame in the scene 0580_00 seems to be inconsistent.. Therefore I wonder if you could provide a corrected GT version?

theNded commented 1 year ago

Sorry for the confusion, I shall re-upload. Just want to confirm -- which coordinate system do you desire? The metric system (from the GT poses), or the normalized system?

Willyzw commented 1 year ago

No problem, the metric one from the GT poses is preferred. Thanks a lot!

By inconsistency in the coordinate system, I actually mean the GT poses of the scene 0580 used by the Manhattan SDF is different from the original scannet dataset. Maybe this is the reason why its numbers in Scene 0580 in your papers Tab 4 are so low..

theNded commented 1 year ago

Good catch, I quickly checked and you seem correct.

Willyzw commented 1 year ago

Thanks for your quick check. Lately I also received the reply (link: https://github.com/zju3dv/manhattan_sdf/issues/57) from the Manhattan SDF author, which clarified that the coordinate inconsistency was caused by a rotation they manually tuned to align with gravity direction. The rotation is as follows:

rotation = np.array([[ 9.81055493e-01, -3.73636465e-04, -1.93726561e-01],
                     [-3.73636465e-04,  9.99992631e-01, -3.82080710e-03],
                     [ 1.93726561e-01,  3.82080710e-03,  9.81048124e-01]])

With this rotation, I can now align their GT mesh with the reconstruction from the GT poses. Therefore, I see no need to re-upload and we can use the GT meshes by Manhattan SDF without any issues.