search

AiuniAI / Unique3D

[NeurIPS 2024] Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image
https://wukailu.github.io/Unique3D/
MIT License
3.11k stars 246 forks source link

Why can ExplicitTarget Optimization solve the Multi-view Inconsistency issue? #44

Closed YuQiao0303 closed 4 months ago

YuQiao0303 commented 5 months ago

Hi, authors. Thank you for this amazing work!

I'm a bit confused by how unique3d solves the multi-view inconsistency issue.

The explicit target looks like finding the color for each 3D point by the weighted sum of corresponding pixels of visible views. Is my understanding correct? Why would this solve the multiview inconsistency issue? It still uses different views, and if the views are inconsistent, why wouldn't it still influence the generated mesh?

wukailu commented 4 months ago

When multiple viewpoints are inconsistent, there is no theoretically optimal solution per se, so it's just a matter of some modifications that allow the iterations to converge to a solution that is good relative to humans. Wavy imperfections tend to appear when the same surface is seen from different viewpoints and in different directions (as this minimises the loss). However, such imperfections are not desired, so averaging the results from different viewpoints ensures that the final surface is relatively smooth.

YuQiao0303 commented 4 months ago

When multiple viewpoints are inconsistent, there is no theoretically optimal solution per se, so it's just a matter of some modifications that allow the iterations to converge to a solution that is good relative to humans. Wavy imperfections tend to appear when the same surface is seen from different viewpoints and in different directions (as this minimises the loss). However, such imperfections are not desired, so averaging the results from different viewpoints ensures that the final surface is relatively smooth.

I see. Thank you!