LituRout / PSLD

Posterior Sampling using Latent Diffusion
124 stars 12 forks source link

the meaning of "x*_0" in Algorithm 2 #6

Closed wangherr closed 5 months ago

wangherr commented 5 months ago

image

What is the meaning of x*_0 or Ax*_0 ? I can't find any definition.

Thanks!

LituRout commented 5 months ago

Hi wangherr,

$x_0$ denotes the unknown groundtruth image and $Ax_0$ denotes the known measurements $y$. In case of noisy inverse problems, $y=Ax_0^* + noise$.

wangherr commented 5 months ago

for line 8,

I can understand it in inpainting.

if in others task, such as Gaussian deblur,

1, will the A A^T x*_0 to be x_0 (groundtruth image)? 2, will the I - A A^T to be 0?

thanks!

LituRout commented 5 months ago

As you have rightly pointed out, line 8 is self-explanatory for inpainting tasks. For Gaussian deblur, it is not clear what $A^T$ would be in this case. However, note that all linear inverse problems can be expressed as inpainting tasks at the cost of a big tedious measurement matrix $A$. Therefore, our algorithm generalizes to other linear inverse tasks.

In practice, we borrow the measurement operators from DPS, which seems to work reasonably well.

Note:- Q1 and Q2 above are incorrect because there is dimension mismatch between $A^T$ and $x_0$. Above answer is for $A^TAx_0$.

wangherr commented 5 months ago

Is 'line 8' only used in inpainting tasks in practice?

LituRout commented 5 months ago

No, it is used for all the inverse tasks where A^T and A are chosen as per the DPS codebase. For inpainting, line 8 is exact and for other inverse tasks, it is an approximation.