wyhuai / DDNM

[ICLR 2023 Oral] Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model
MIT License
1.16k stars 83 forks source link

Error when using my own image #72

Open munib94 opened 6 months ago

munib94 commented 6 months ago

Hi,

When I execute the following command on a blurry image:

python main.py --ni --config imagenet_256_cc.yml --path_y ../../../artifacts_dataset/small_dataset/val/0145.jpg --eta 0.85 --deg "deblur_gauss" --deg_scale 2.0 --sigma_y 0.1 -i demo

I get the following error:

INFO - main.py - 2024-05-02 02:24:38,466 - Using device: cuda
2.21GB [02:55, 12.6MB/s]                                                                                                                                                                                                                                                                                   
ERROR - main.py - 2024-05-02 02:27:39,674 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 161, in sample
    model.load_state_dict(torch.load(ckpt, map_location=self.device))
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2189, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for UNetModel:
        Missing key(s) in state_dict: "label_emb.weight". 

Can you tell me what I am doing wrong here? I am using the pretrained model for restoring general images.

wyhuai commented 6 months ago

Hi, we didn't implement the code for noisy deblur.

munib94 commented 6 months ago

Ok, in that case, I tried the following:

python main.py --ni --simplified --config imagenet_256_cc.yml --path_y ../artifacts_dataset/0050_siteA10_single_camera_ground_deblur_75/input_images/image_000008.jpg --eta 0.85 --deg "sr_averagepooling" --deg_scale 1.0 --sigma_y 0.1 -i diy

and got this error:

INFO - main.py - 2024-05-03 17:51:48,548 - Using device: cuda
Run Simplified DDNM, without SVD. 100 sampling steps. travel_length = 1, travel_repeat = 1. Task: sr_averagepooling.
building dataset from exp/imagenet_val_1k.txt
read meta done
Dataset has size 8
/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/utils/data/dataloader.py:558: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
args.deg: sr_averagepooling
Start from 0
  0%|                                                                                                                                                                                                                                                              | 0/100 [00:00<?, ?it/s]
  0%|                                                                                                                                                                                                                                                                | 0/8 [00:00<?, ?it/s]
ERROR - main.py - 2024-05-03 17:51:55,236 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 199, in sample
    self.simplified_ddnm_plus(model, cls_fn)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 358, in simplified_ddnm_plus
    et = model(xt, t)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 183, in forward
    return self.module(*inputs[0], **module_kwargs[0])
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/user/.pyenv/versions/ddnm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/MSU/Research/DDNM/guided_diffusion/unet.py", line 644, in forward
    assert (y is not None) == (
AssertionError: must specify y if and only if the model is class-conditional

Can you tell me what I am missing here?

wyhuai commented 6 months ago

Did you use the class-conditional model without providing conditions?

munib94 commented 6 months ago

I have both 256x256_classifier.pt and 256x256_diffusion_uncond.pt in the path DDNM/exp/logs/imagenet. If I remove 256x256_classifier.pt, I get the error,

INFO - main.py - 2024-05-05 22:06:05,422 - Using device: cuda
ERROR - main.py - 2024-05-05 22:06:10,654 - Traceback (most recent call last):
  File "main.py", line 166, in main
    runner.sample(args.simplified)
  File "/home/munib/MSU/Research/DDNM/guided_diffusion/diffusion.py", line 172, in sample
    'https://openaipublic.blob.core.windows.net/diffusion/jul-2021/%dx%d_classifier.pt' % image_size, ckpt)
TypeError: not enough arguments for format string

I am not sure which conditions you are referring to? Can you direct me to where in the code I can find the conditions it is expecting?

wyhuai commented 6 months ago

Can you run evaluation.sh successfully?

munib94 commented 6 months ago

Yes, it runs successfully. I also think there is a bug when I run the same command without the --simplified option. The path to the input images defaults to DDNM/exp/datasets/imagenet/imagenet instead of the path that I provide.

wyhuai commented 6 months ago

Errors may occur when using personal configurations. I recommend trying it step by step based on the reference evaluation.sh