Question with regards to results on Indoor6 dataset

[AustrianOakvn]

Hi, thank you for great work. I try to use ACE on the Indoor6 dataset provided by the following: https://github.com/microsoft/SceneLandmarkLocalization. However, the results are not so good, the translation error reach 1 meter and the rotation error can be up to 100 degrees. Because indoor6 dataset is collected at different time and day, it contains high illumination variations. Could ACE work in such cases, are there any configuration that i missed?

[ebrach]

Hi,

we did not try the Indoor-6 dataset but the images you share look challenging for ACE. The Indoor-6 paper contains numbers for DSAC. They ran DSAC in "poses+3d-model" mapping mode which is a slightly easier task than ACE's "poses-only" mapping mode. That said, I think one can expect similar performance of ACE and DSAC* if ACE mapping succeeds. Whether or not ACE mapping succeeds can be somewhat checked via the visualisation capabilities of ACE.

I would recommend to check the visualisations and see whether the mapping camera poses look OK (to rule out any dataset conversion issue), and whether ACE learns plausible scene geometry.

Best, Eric

[AustrianOakvn]

Thank you for your response, after checking the code and testing ACE with all scene in the indoor6, the performance was just like you said, it is quite close to DSAC*. translation error (cm)/rotation error (deg)		scene1	scene2a	scene3	scene4a	scene5	scene6
DSAC*	12.3/2.06	7.9/0.9	13.1/2.34	3.7/0.95	40.7/6.72	6.0/1.4
ACE	13.6/2.1	6.8/0.7	8.1/1.3	4.8/0.9	14.7/2.3	6.1/1/1

Again, ACE is really impressive in term of speed and memory efficiency but I wonder for example what can be use to improve ACE on such cases mentioned above with high illumination and drastic environmental changes.

[ebrach]

I think the feature backbone of ACE needs to be improved to be more robust to such condition changes.

I will close this issue since, I think, your original question was answered.