zsyOAOA / DifFace

DifFace: Blind Face Restoration with Diffused Error Contraction (TPAMI, 2024)
Other
628 stars 42 forks source link

About the guidance function of estimator #12

Closed chensming closed 1 year ago

chensming commented 1 year ago

Hi, I am interested in your nice work, but after reading your paper, I donot understand the way how estimator work. Therefore, I read your code, and I suppose that the estimator(swinIR) is used to predict the x0 by low quality image y0, and add noise like diffusion forward process do?

https://github.com/zsyOAOA/DifFace/blob/35d91a1c029b8a910b5a66684e0b51862bbe8e34/sampler.py#L247

im_hq = self.model_ir(y0)

and https://github.com/zsyOAOA/DifFace/blob/35d91a1c029b8a910b5a66684e0b51862bbe8e34/sampler.py#L259

yt = self.diffusion.q_sample(
             x_start=post_fun(im_hq),
             t=torch.tensor([start_timesteps,]*im_hq.shape[0], device=device),
)

finally sample by ddpm https://github.com/zsyOAOA/DifFace/blob/35d91a1c029b8a910b5a66684e0b51862bbe8e34/sampler.py#L279

sample = self.diffusion.p_sample_loop(
                    self.model,
                    shape=yt.shape,
                    noise=yt,
                    start_timesteps=start_timesteps,
                    clip_denoised=True,
                    denoised_fn=None,
                    model_kwargs=None,
                    device=None,
                    progress=False,
 )

The script I run is as followings. Did I miss any details? ^_^

python inference_difface.py --aligned --in_path testdata/cropped_faces --out_path result/testdata --gpu_id 2
zsyOAOA commented 1 year ago

The diffused estimator provides a initial estimation of x_0, then diffuse to one specific timestep, finally reverse back.

chensming commented 1 year ago

The diffused estimator provides a initial estimation of x_0, then diffuse to one specific timestep, finally reverse back.

ok, thanks!