search

3DTopia / LGM

[ECCV 2024 Oral] LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation.
https://me.kiui.moe/lgm/
MIT License
1.69k stars 115 forks source link

Not able to generate consistent assets #13

Open rajatprasun opened 9 months ago

rajatprasun commented 9 months ago

Hi, thank you for putting this together. The results shown in the demo are amazing. But for some reason, I am not able to create clean assets. The generated multi views look good but the 3D assets don't look good. I tried one of the demo prompts as well.

Prompt : a motorbike Multiview : image 3D video : https://github.com/3DTopia/LGM/assets/20456046/471c54ac-af6f-4651-8515-15cfff679484

Also, the exported meshes do not have color or texture information.

ashawkey commented 9 months ago

@rajatprasun Hi, unfortunately MVDream can be unstable sometimes. You can try with different random seeds, and examine the mv images first. The mesh is exported in GLB, which is supposed to have texture. You can load it with blender or other 3D viewers to check.

rajatprasun commented 9 months ago

Hey, I tried with different seeds as well and increased the iteration to 500 as well. The multi view look great but the 3D assets in the generated video are smeared or have noise.

For the texture part, I Imported the .ply file in blender for the bird from image-to-3d example and it looks like this: image

I do not see the texture from the image. Is this expected?

ashawkey commented 9 months ago

@rajatprasun Surely the model still have some limitations for complex prompts... This ply file is the 3D Gaussians instead of the final mesh. You need to run convert.py on this ply file to get the final textured mesh. Since it takes ~1 minute to do this conversion, it's not included in the gradio demo.

rajatprasun commented 9 months ago

Hey thanks for the info. After running the convert.py on this ply file. I am able to generate the textured mesh. image