Using blurred images only, how to train your code??

[edwardcho]

Hello sir, I have interesting image deblurring and i searched your code.

I checked your code. I found that your network needs paired images (blurred image vs sharp image vs etc. ) I want to know input-data information.

This is your config.py.

    # data dir
    # config.data_offset = '/data1/junyonglee/defocus_deblur'
    config.data_offset = 'datasets/defocus_deblur'
    config.c_path = os.path.join(config.data_offset, 'DPDD/train_c')
    config.l_path = os.path.join(config.data_offset, 'DPDD/train_l')
    config.r_path = os.path.join(config.data_offset, 'DPDD/train_r')

I have blurred images only. Can i try to train your code??

Thanks, Edward Cho.

[codeslake]

Hi, Edward.

If you don't have ground-truth sharp images that are paired with the blurry images that you have, it is basically impossible to train the network at the moment unless you don't use an unsupervised training scheme such as GAN, which we did not use in our method.

If you have ground-truth pairs, but If you don't have dual-pixel stereo images (DPDD/train_l, and DPDD/train_r), do the followings (I'm pretty sure that it will not work at once. Share errors if you do the followings):

• First, copy models/archs/IFAN.py to a file with a new name models/archs/IFAN_no_DME.py.
• Next, you need to remove the part of the code in the forward function of the network (here, here, and here) to not handle R, which indicates the right-hand side of dual-pixel stereo images.
• Next, you need to modify the part of the code taking dual-pixel disparity to not take the disparity information.
  • Modify input channels from 2 * ch4 to ch4 (here)
  • Modify this line to f = self.conv4_4(f)
• Then, train the network with the following code:

  CUDA_VISIBLE_DEVICES=0,1,2,3 python -B -m torch.distributed.launch --nproc_per_node=4 --master_port=9000 run.py \
            --is_train \
            --mode C_only \
            --config config_IFAN \
            --trainer trainer \
            --network IFAN_no_DME \
            -b 2 \
            -th 8 \
            -dl \
            -ss \
            -dist

• After training is done, test the network with the following code:

  CUDA_VISIBLE_DEVICES=0 python run.py --mode C_only --network IFAN_no_DME --config config_IFAN --data DPDD --ckpt_sc

[edwardcho]

Hello Sir,

Thank you for your response quickly.

If you don't mind, please tell me another networks for my cases (only blurred images).

Thanks, Edward Cho.

[codeslake]

Hi, Edward.

If you don't have ground-truth images, search for papers or repos with the keyword, unsupervised, in here. As far as know, you will need unpaired sharp images though.

[edwardcho]

Hello Sir,

Unfortunately, I can't have any sharp images as GT. So I think that I can't try networks of unsupervised learning.

Thanks, Edward Cho.

[codeslake]

Why don't you use GT sharp images of any deblurring datasets?

[edwardcho]

Hello Sir,

From my customers, I didn't get sharp images that are matched with blurred images.

Thanks, Edward Cho.

[codeslake]

That's why I suggested unsupervised learning, which does not need paired sharp images. You can use unmatched sharp images, which you can obtain easily from any supervised deblurring datasets, to train a network to deblur your blurry images in the unsupervised setting.

[edwardcho]

Hello Sir,

Thanks to your advice, I was inspired. Thanks for your great advice.

Thanks, Edward Cho.

[hust-lidelong]

Hi, Edward.

If you don't have ground-truth sharp images that are paired with the blurry images that you have, it is basically impossible to train the network at the moment unless you don't use an unsupervised training scheme such as GAN, which we did not use in our method.

If you have ground-truth pairs, but If you don't have dual-pixel stereo images (DPDD/train_l, and DPDD/train_r), do the followings (I'm pretty sure that it will not work at once. Share errors if you do the followings):

• First, copy models/archs/IFAN.py to a file with a new name models/archs/IFAN_no_DME.py.
• Next, you need to remove the part of the code in the forward function of the network (here, here, and here) to not handle R, which indicates the right-hand side of dual-pixel stereo images.

• Next, you need to modify the part of the code taking dual-pixel disparity to not take the disparity information.

  • Modify input channels from 2 * ch4 to ch4 (here)
  • Modify this line to f = self.conv4_4(f)
• Then, train the network with the following code:

  CUDA_VISIBLE_DEVICES=0,1,2,3 python -B -m torch.distributed.launch --nproc_per_node=4 --master_port=9000 run.py \
            --is_train \
            --mode C_only \
            --config config_IFAN \
            --trainer trainer \
            --network IFAN_no_DME \
            -b 2 \
            -th 8 \
            -dl \
            -ss \
            -dist

• After training is done, test the network with the following code:

  CUDA_VISIBLE_DEVICES=0 python run.py --mode C_only --network IFAN_no_DME --config config_IFAN --data DPDD --ckpt_sc

Dear author, following your models/archs/IFAN_no_DME.py, IFAN only contains three parts: Filter Encoder, Filter Predictor, IAC? It is the same as your supplementary article: Figure 15. Network architectures used for the ablation study ((c) the model with filter predictor and IAC (Table 6))? Is my understanding correct? Now, I only have clean images. Do I need to make blurred images？ Can you teach me how to make image pairs for training with IFAN? Thanks very much!

[codeslake]

Dear author, following your models/archs/IFAN_no_DME.py, IFAN only contains three parts: Filter Encoder, Filter Predictor, IAC? It is the same as your supplementary article: Figure 15. Network architectures used for the ablation study ((c) the model with filter predictor and IAC (Table 6))? Is my understanding correct?

Yes, it is for data without dual-pixel stereo images, as you will not be able to train IFAN with the disparity estimation task.

Now, I only have clean images. Do I need to make blurred images？ Can you teach me how to make image pairs for training with IFAN? Thanks very much!

I am afraid if I understand correctly. Why are you trying to do defocus deblurring if you have clean images only? If it is really true that you have clean images only, yes, you will need to generate defocused images somehow. For good deblurring quality, you will have to apply spatially varying defocus blur considering the depth of a scene.

[hust-lidelong]

Dear author, following your models/archs/IFAN_no_DME.py, IFAN only contains three parts: Filter Encoder, Filter Predictor, IAC? It is the same as your supplementary article: Figure 15. Network architectures used for the ablation study ((c) the model with filter predictor and IAC (Table 6))? Is my understanding correct?

Yes, it is for data without dual-pixel stereo images, as you will not be able to train IFAN with the disparity estimation task.

Now, I only have clean images. Do I need to make blurred images？ Can you teach me how to make image pairs for training with IFAN? Thanks very much!

I am afraid if I understand correctly. Why are you trying to do defocus deblurring if you have clean images only? If it is really true that you have clean images only, yes, you will need to generate defocused images somehow. For good deblurring quality, you will have to apply spatially varying defocus blur considering the depth of a scene.

Sorry I didn't explain clearly. I want to explore the possibility of defocus deblurring in an industrial dataset. At present, the dataset only has clear images and no defocused images. In the actual operation stage of the equipment, the problem of defocus blurring will be encountered. So could you share some codes or URLs for making defocused images, based on clear images? I am still a novice, thank you very much！

[codeslake]

Sorry I didn't explain clearly. I want to explore the possibility of defocus deblurring in an industrial dataset. At present, the dataset only has clear images and no defocused images. In the actual operation stage of the equipment, the problem of defocus blurring will be encountered. So could you share some codes or URLs for making defocused images, based on clear images? I am still a novice, thank you very much！

I see! I would start from this article, and later papers that cited the paper.

[hust-lidelong]

Sorry I didn't explain clearly. I want to explore the possibility of defocus deblurring in an industrial dataset. At present, the dataset only has clear images and no defocused images. In the actual operation stage of the equipment, the problem of defocus blurring will be encountered. So could you share some codes or URLs for making defocused images, based on clear images? I am still a novice, thank you very much！

I see! I would start from this article, and later papers that cited the paper.

Thanks for your help! This article doesn't seem to have open-source code, do you know of other open-source code for making image pairs? And after I make paired dataset, how to set up options for training, I am very confused. For example, https://github.com/codeslake/IFAN/blob/main/configs/config.py#L84-L88 and https://github.com/codeslake/IFAN/blob/main/configs/config.py#L117 and https://github.com/codeslake/IFAN/blob/main/configs/config.py#L129 and https://github.com/codeslake/IFAN/blob/main/configs/config.py#L135 How to choose?

[codeslake]

Thanks for your help! This article doesn't seem to have open-source code, do you know of other open-source code for making image pairs?

I'm not 100% sure, but I believe it had a code that runs on Ubuntu. Or, search for open source the other papers that cited the paper that I suggested.

And after I make paired dataset, how to set up options for training, I am very confused.

I would set up the dataset structure as the same structure as the DPDD dataset. Rest is easy, remove the path related to stereo images (e.g., l_path, r_path). For the rest, I can't help with every detail. You will have to work with your own dataset. Make sure to use the model without the disparity map estimation module tough.

[hust-lidelong]

Thanks for your help! This article doesn't seem to have open-source code, do you know of other open-source code for making image pairs?

I'm not 100% sure, but I believe it had a code that runs on Ubuntu. Or, search for open source the other papers that cited the paper I suggested.

And after I make paired dataset, how to set up options for training, I am very confused.

I would set up the dataset structure as the same structure as the DPDD dataset. Rest is easy, remove the path related to stereo images (e.g., l_path, r_path). For the rest, I can't help with every detail. You will have to work with your own dataset. Make sure to use the model without the disparity map estimation module tough.

Thanks very much, you really help me a lot. I will follow your prompts to set it up.

[codeslake]

@hust-lidelong, Just to make sure, If you decide to remove the path related to stereo images (e.g., l_path, r_path), you will have to modify the dataset file and remove all lines related with l_path, r_path, as well as lines related with L and R in trainer, model, eval files.

If you wanna go with an easy workaround, just leave all the code as it is, but make l_path and r_path the same as c_path in the config file.

[hust-lidelong]

@hust-lidelong, Just to make sure, If you decide to remove the path related to stereo images (e.g., l_path, r_path), you will have to modify the dataset file and remove all lines related with l_path, r_path, as well as lines related with L and R in trainer, model, eval files.

If you wanna go with an easy workaround, just leave all the code as it is, but make l_path and r_path the same as c_path in the config file.

Thank you for your welcome reminder. So if I wanna go with an easy workaround, I make l_path and r_path the same as c_path. But I find https://github.com/codeslake/IFAN/blob/main/configs/config.py#L81 with the note "VALIDATION". Does this mean it is the address of validation set? Where do I modify the address of the training set and validation set for my own dataset? Thanks~

[codeslake]

Thank you for your welcome reminder. So if I wanna go with an easy workaround, I make l_path and r_path the same as c_path. But I find https://github.com/codeslake/IFAN/blob/main/configs/config.py#L81 with the note "VALIDATION". Does this mean it is the address of validation set? Where do I modify the address of the training set and validation set for my own dataset? Thanks~

Yes, it is the path for the validation set. What I meant to have l_path and r_path the same as c_path was:

  config.VAL.c_path = os.path.join(config.data_offset, 'DPDD/val_c')
  config.VAL.l_path = os.path.join(config.data_offset, 'DPDD/val_c')
  config.VAL.r_path = os.path.join(config.data_offset, 'DPDD/val_c')

  config.c_path = os.path.join(config.data_offset, 'DPDD/train_c')
  config.l_path = os.path.join(config.data_offset, 'DPDD/train_c')
  config.r_path = os.path.join(config.data_offset, 'DPDD/train_c')

  config.EVAL.c_path = os.path.join(config.data_offset, 'DPDD/eval_c')
  config.EVAL.l_path = os.path.join(config.data_offset, 'DPDD/eval_c')
  config.EVAL.r_path = os.path.join(config.data_offset, 'DPDD/eval_c')

[hust-lidelong]

Thank you for your welcome reminder. So if I wanna go with an easy workaround, I make l_path and r_path the same as c_path. But I find https://github.com/codeslake/IFAN/blob/main/configs/config.py#L81 with the note "VALIDATION". Does this mean it is the address of validation set? Where do I modify the address of the training set and validation set for my own dataset? Thanks~

Yes, it is the path for the validation set. What I meant to have l_path and r_path the same as c_path was:

  config.VAL.c_path = os.path.join(config.data_offset, 'DPDD/val_c')
  config.VAL.l_path = os.path.join(config.data_offset, 'DPDD/val_c')
  config.VAL.r_path = os.path.join(config.data_offset, 'DPDD/val_c')

  config.c_path = os.path.join(config.data_offset, 'DPDD/train_c')
  config.l_path = os.path.join(config.data_offset, 'DPDD/train_c')
  config.r_path = os.path.join(config.data_offset, 'DPDD/train_c')

  config.EVAL.c_path = os.path.join(config.data_offset, 'DPDD/eval_c')
  config.EVAL.l_path = os.path.join(config.data_offset, 'DPDD/eval_c')
  config.EVAL.r_path = os.path.join(config.data_offset, 'DPDD/eval_c')

I finally understand, Thanks！ I wish you success in your research and more and more papers.

[codeslake]

Thank you 👍