search

wyhuai / DDNM

[ICLR 2023 Oral] Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model
MIT License
1.16k stars 83 forks source link

How do I make the inpainted images less blurry? #35

Open davidboxboro opened 1 year ago

davidboxboro commented 1 year ago

When I run the unconditional ImageNet model on sample ImageNet validation images, the inpainted portion is often very blurry. Is there a way to fix this?

wyhuai commented 1 year ago

Hi, The performance ceiling depends on the model's generative power. So I recommended using the conditional model for ImageNet Inpainting.

davidboxboro commented 1 year ago

Thank you, that works!

ferdifdi commented 1 year ago

I want to ask something. I use imagenet_256_cc.yml conditional simplified inpainting way, but there is error. But, when I try the SVD one, it can.

ERROR - main.py - 2023-04-23 14:03:16,556 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 284, in sample
    self.simplified_ddnm_plus(model, cls_fn)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 470, in simplified_ddnm_plus
    et = model(xt, t)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/workspace/DDNM/guided_diffusion/unet.py", line 644, in forward
    assert (y is not None) == (
AssertionError: must specify y if and only if the model is class-conditional

Thank you

wyhuai commented 1 year ago

I want to ask something. I use imagenet_256_cc.yml conditional simplified inpainting way, but there is error. But, when I try the SVD one, it can.

ERROR - main.py - 2023-04-23 14:03:16,556 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 284, in sample
    self.simplified_ddnm_plus(model, cls_fn)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 470, in simplified_ddnm_plus
    et = model(xt, t)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/workspace/DDNM/guided_diffusion/unet.py", line 644, in forward
    assert (y is not None) == (
AssertionError: must specify y if and only if the model is class-conditional

Thank you

It seems that you did not provide condition y, usually a class number.

ferdifdi commented 1 year ago

How to provide y? When I provide number, the error say int has no attribute shape. When I add --class 950 in command, it still didnt work like in hq demo.

ERROR - main.py - 2023-04-23 14:17:28,881 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 284, in sample
    self.simplified_ddnm_plus(model, cls_fn)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 470, in simplified_ddnm_plus
    et = model(xt, t)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/workspace/DDNM/guided_diffusion/unet.py", line 652, in forward
    assert y.shape == (x.shape[0],)
AttributeError: 'int' object has no attribute 'shape'
wyhuai commented 1 year ago

How to provide y? When I provide number, the error say int has no attribute shape. When I add --class 950 in command, it still didnt work like in hq demo.

ERROR - main.py - 2023-04-23 14:17:28,881 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 284, in sample
    self.simplified_ddnm_plus(model, cls_fn)
  File "/workspace/DDNM/guided_diffusion/diffusion.py", line 470, in simplified_ddnm_plus
    et = model(xt, t)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/workspace/DDNM/guided_diffusion/unet.py", line 652, in forward
    assert y.shape == (x.shape[0],)
AttributeError: 'int' object has no attribute 'shape'

Maybe you need to define a tensor y, not an int number. what about y=torch.tensor([950]), with batch size 1?