search

chensjtu / GaussianObject

GaussianObject: High-Quality 3D Object Reconstruction from Four Views with Gaussian Splatting (SIGGRAPH Asia 2024, TOG)
887 stars 54 forks source link

Can We Use MVDream MultiView diffusion model? #24

Open joshkiller opened 7 months ago

joshkiller commented 7 months ago

Hello and congratulations for the remarquable work. I was wondering if there are a way to feed your model with 4 images from MVDream automatically to have a kind of text to 3D generation? Maybe my question doesn't have any sense yeah because I'm beginner with research and this one is my first master internship project

GaussianObject commented 7 months ago

Hello,

Firstly, congratulations on starting your journey in research with your master internship project, and thank you for your interest in our work! GaussianObject is primarily designed for reconstruction tasks rather than generation. With regards to using images generated by MVDream for 3D reconstruction, it's important to note that the consistency of 3D features across generated images is uncertain. Additionally, the inherent diversity of text-to-image models may pose challenges for camera pose estimation methods.

If your goal is to achieve text-to-3D generation, you might want to explore other avenues specifically tailored for this purpose. One such method that could align with your interests is GaussianDreamer, which might better suit the requirements of a text-to-3D pipeline.

We wish you the best of luck with your project and are excited to see where your research takes you!

joshkiller commented 7 months ago

@GaussianObject Thank you for your encouraging words and support! I'm excited to explore this avenue further and see where it leads. Your input has definitely provided me with a valuable direction to consider. Thanks again for your assistance, and I look forward to continuing this journey.