search

wyysf-98 / CraftsMan

CraftsMan: High-fidelity Mesh Generation with 3D Native Diffusion and Interactive Geometry Refiner
https://craftsman3d.github.io/
430 stars 22 forks source link

Clarification about `zero_uncond_embeds` hyperparameter #27

Open rfeinman opened 2 months ago

rfeinman commented 2 months ago

Hi,

What is the correct value for the zero_uncond_embeds hyperparameter of your released diffusion model checkpoint clip-mvrgb-modln-l256-e64-ne8-nd16-nl6? In the config file included with this github repo, the value is set to False. However in the huggingface repo, it is set to True. Which is correct?

config file in this github repo:

https://github.com/wyysf-98/CraftsMan/blob/9c836e201a122b1f89fc51b3152de870b2ef6ba6/configs/image-to-shape-diffusion/clip-mvrgb-modln-l256-e64-ne8-nd16-nl6.yaml#L63

config file in huggingface repo:

https://huggingface.co/wyysf/CraftsMan/blob/main/image-to-shape-diffusion/clip-mvrgb-modln-l256-e64-ne8-nd16-nl6/config.yaml#L68

wyysf-98 commented 1 month ago

Hi, this is a historical issue. The two correspond to different ckpts (aligned-vae and w/o aligned). The ckpt in hf is aligned-vae with True zero_uncond_embeds and we will unified it in the future. Set zero_uncond_embeds maybe better in my practice.

Moreover, we will release a dit-based version soon, which will greatly improve performance. Please stay tuned. (The input is the 1st col, followed with 4 rendered normals with predicted mesh)

image

rfeinman commented 1 month ago

Hi @wyysf-98

FYI the huggingface repo contains two separate configs:

  1. aligned VAE variant
  2. w/o aligned VAE variant

According to these configs, zero_uncond_embeds=True in both variants. Is this a mistake?