search

SUDO-AI-3D / zero123plus

Code repository for Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model.
Apache License 2.0
1.69k stars 115 forks source link

Depth ControlNet not generating expected output #95

Open LIU-Yuxin opened 1 week ago

LIU-Yuxin commented 1 week ago

Thank you for the sharing the great work. I am currently using this method for generating textures given a mesh, by first rendering the depth images, and then generate the views with an additional reference image, similar to the one in the depth controlnet example, with depth images normalized as in #40. However, the quality of the generated image is not comparable to the image without controlnet, as shown below. Could you please let me know if there are any fix to this?

Input reference image: sneakers Output without controlnet: output_wo Rendered depth image (I assume the alpha channel is used for mask, and the opaque region should be normalized as in #40 ): depth Output with controlnet: output_w

I have tried to adjust the size of the object, and also the weight of the controlnet, but did not generate result similar to the version without controlnet.

Similar issue also applies to face model and image. face1 w/o output w/ output

Looking forward to hear your reply. Thank you!

eliphatfs commented 5 days ago

I think for the sneaker the control pose is different than the prior pose which may be causing a problem. For the head i am not sure.

joeybchen commented 11 hours ago

Hi @LIU-Yuxin Can I know how to get the depth maps for the objaverse data like you? Many thanks!