Replacing the Stable Diffusion model for a user-trained model

[raymondgp]

Thank you for making this available to all.

I have not an issue per say as your code is running successfully on my Windows 10/RTX3060 12GB, but is there a way to run the 3d generation based on a user-provided, trained, stable diffusion model?

I tried a simple replacement for what I would consider a working trained (dreambooth) SD model but I received the error at below (yes, I just renamed my model to sd-v1-5.ckpt):

Is there a way to modify the adapt_sd.py script to run custom models?

============error============================== Loading model from ..\release\diffusion_ckpts\stable_diffusion\sd-v1-5.ckpt Traceback (most recent call last): File "D:\3dconversion\sjc\run_sjc.py", line 297, in dispatch(SJC) File "D:\3dconversion\sjc\my\config.py", line 76, in dispatch mod.run() File "D:\3dconversion\sjc\run_sjc.py", line 77, in run model = getattr(self, family).make() File "D:\3dconversion\sjc\run_img_sampling.py", line 39, in make model = StableDiffusion(**args) File "D:\3dconversion\sjc\adapt_sd.py", line 90, in init self.model, H, W = load_sd1_model(self.checkpoint_root()) File "D:\3dconversion\sjc\adapt_sd.py", line 59, in load_sd1_model model = load_model_from_config(config, str(ckpt_fname)) File "D:\3dconversion\sjc\adapt_sd.py", line 38, in load_model_from_config sd = pl_sd["state_dict"] KeyError: 'state_dict'

[raymondgp]

Thank you for making this available to all.

I have not an issue per say as your code is running successfully on my Windows 10/RTX3060 12GB, but is there a way to run the 3d generation based on a user-provided, trained, stable diffusion model?

I tried a simple replacement for what I would consider a working trained (dreambooth) SD model but I received the error at below (yes, I just renamed my model to sd-v1-5.ckpt):

Is there a way to modify the adapt_sd.py script to run custom models?

============error============================== Loading model from ..\release\diffusion_ckpts\stable_diffusion\sd-v1-5.ckpt Traceback (most recent call last): File "D:\3dconversion\sjc\run_sjc.py", line 297, in dispatch(SJC) File "D:\3dconversion\sjc\my\config.py", line 76, in dispatch mod.run() File "D:\3dconversion\sjc\run_sjc.py", line 77, in run model = getattr(self, family).make() File "D:\3dconversion\sjc\run_img_sampling.py", line 39, in make model = StableDiffusion(args) File "D:\3dconversion\sjc\adapt_sd.py", line 90, in init** self.model, H, W = load_sd1_model(self.checkpoint_root()) File "D:\3dconversion\sjc\adapt_sd.py", line 59, in load_sd1_model model = load_model_from_config(config, str(ckpt_fname)) File "D:\3dconversion\sjc\adapt_sd.py", line 38, in load_model_from_config sd = pl_sd["state_dict"] KeyError: 'state_dict'

Nevermind above. I saw my error. I modified adapt_sd.py to use the checkpoint of my model as the state_dict, this made the model load successfully. I'm currently generating multiple views of a fully customized instance.

You sir and the people that wrote the paper are legendary, this is going to revolutionize 3d modeling in the supra short term.

[raymondgp]

found solution to my own silly error