search

LuChengTHU / dpm-solver

Official code for "DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps" (Neurips 2022 Oral)
MIT License
1.56k stars 121 forks source link

Control net #41

Open jianghuyihei opened 1 year ago

jianghuyihei commented 1 year ago

Can dpm-solver used for control-net?

LuChengTHU commented 1 year ago

Of course. Please check https://github.com/AUTOMATIC1111/stable-diffusion-webui , where all the samplers starting with "DPM" are dpm-solver (with different settings). As far as I known, the best are "DPM++2M" and "DPM++2M Karras", which are 2nd-order multistep dpm-solver++, w/o Karras' time step settings.

jianghuyihei commented 1 year ago

Of course. Please check https://github.com/AUTOMATIC1111/stable-diffusion-webui , where all the samplers starting with "DPM" are dpm-solver (with different settings). As far as I known, the best are "DPM++2M" and "DPM++2M Karras", which are 2nd-order multistep dpm-solver++, w/o Karras' time step settings.

Sorry to bother you, your results are very exciting, and I have achieved good results in unconditional generation, but I am currently encountering a problem. I trained a diffusion model by myself,it is a conditional diffsuion,can be described as F(x,t,c),the condition added as control-net(It can be analogized as a condition module),but my generative quality is poor,why?There is my code. model_fn = model_wrapper( eps_model, noise_schedule, guidance_type ="classifier-free", condition = condition, model_type="noise", # or "x_start" or "v" or "score" ) dpm_solver = DPM_Solver(model_fn, noise_schedule, algorithm_type="dpmsolver++",correcting_x0_fn="dynamic_thresholding") x_T = torch.randn(sample_num,3,512,512).to(device) x_sample = dpm_solver.sample( x_T, steps=50, order=3, skip_type="time_uniform", method="multistep", )

LuChengTHU commented 1 year ago

Hi @jianghuyihei , for conditional sampling, please use order=2.

Moreover, could you please give me more details? e.g., can order=1 work? (which is equivalent to DDIM); what is your guidance scale? What does the image look like?

jianghuyihei commented 1 year ago

Hi @jianghuyihei , for conditional sampling, please use order=2.

Moreover, could you please give me more details? e.g., can order=1 work? (which is equivalent to DDIM); what is your guidance scale? What does the image look like? Ok,fine,I do not use guidance scale,and while order = 1,step =1000,result is also bad.My date is a road inraster,size of 512*512.Unconditionally generated very well, perhaps because I added conditions?How to deal with it. My condition added as control net(i transform it as a conditional mouldule into the unet)

LuChengTHU commented 1 year ago

@jianghuyihei I think the bug is not related to the solver, because order=1 is DDIM.

How do you train your conditional model? Maybe you should check the code of control net to figure out why your conditional model is bad...