About shap-e guidance.

[Colezwhy]

Hi, sorry for bothering, but I wonder if there are some differences between the original GaussianDreamer repo and this threestudio extension? Since I found that in the original one they used a finetuned model, here which model is used? Thank you for that.

[jimmychuang95]

@Colezwhy You can simply put the pretrained shap-e model from GaussianDreamer, "shapE_finetuned_with_330kdata.pth", into the folder /custom/threestudio-shap-e/shap-e/cache/.

And don't forget to change line 26 in /custom/threestudio-shap-e/shap_e.py from

finetune_model_name: str = "finetune_model_name.pt"

to

finetune_model_name: str = "shapE_finetuned_with_330kdata.pth"

It will automatically run with the same shap-e model using the code below:

if os.path.exists(os.path.join(self.cfg.cache_dir, self.cfg.finetune_model_name)):
        model.load_state_dict(torch.load(os.path.join(self.cfg.cache_dir, self.cfg.finetune_model_name), map_location=device)['model_state_dict'])

However, I wonder why the initialized shape looks different when running with MVDream and StableDiffusion, even though they have the same seed.

Still figuring it out.

[Colezwhy]

@Colezwhy You can simply put the pretrained shap-e model from GaussianDreamer, "shapE_finetuned_with_330kdata.pth", into the folder /custom/threestudio-shap-e/shap-e/cache/.

And don't forget to change line 26 in /custom/threestudio-shap-e/shap_e.py from

finetune_model_name: str = "finetune_model_name.pt"

to

finetune_model_name: str = "shapE_finetuned_with_330kdata.pth"

It will automatically run with the same shap-e model using the code below:

if os.path.exists(os.path.join(self.cfg.cache_dir, self.cfg.finetune_model_name)):
        model.load_state_dict(torch.load(os.path.join(self.cfg.cache_dir, self.cfg.finetune_model_name), map_location=device)['model_state_dict'])

However, I wonder why the initialized shape looks different when running with MVDream and StableDiffusion, even though they have the same seed.

Still figuring it out.

Thank you for that. Are you using LRM initialization when using MVDream guidance? I do not think there will be anything different in the initialized shape when using different guidance.