How to escape the local minima when searching values of decision variables?

[goldoak]

Hi, thank you for sharing this excellent work! I was testing the code on my own dataset, which consists of some synthetic models without a background. During the search for values of the decision variables, I found that the results are highly dependent on random initialization. If the initialization is good, the process quickly converges to the ground truth. However, most of the time, the initialization is not ideal, making it difficult to converge. Additionally, the non-linear optimization following the search doesn't seem to help much with this problem; it often yields worse results and stops after the first iteration. Could you please share how you address this issue when working with real images?

[vevenom]

Thanks for your kind words!

You are absolutely right. Please note that we have currently only released the coordinate descent baseline. Because of its greedy nature, coordinate descent is bound to end up in a bad local minima, which is subject to initialization like you already pointed out.

We will release our implementations of stochastic algorithms that are much more robust to initialisation, together with our experiments on the ScanNet dataset. The release is planned for this December / January.

[goldoak]

I really appreciate this work and can't wait to try the complete code. Do you have plans to extend it to include more complex structures or additional categories?

[vevenom]

We are currently considering how to make our programs more generalizable to reconstructing object details. New categories are not the priority although I understand this aspect is important for full scene understanding. Stay tuned :)