How to train my own dataset

hust-lidelong commented 2 years ago

Hi! Dear author: I currently have some 1280x960 clear industrial product images. Now I need to solve the problem of defocus blur during equipment operation. Do you know how to make a dataset and train it with your network？ Thanks!

codeslake commented 2 years ago

Hi, @hust-lidelong.

Please refer to this answer!

hust-lidelong commented 2 years ago

Hi, @hust-lidelong.

Please refer to this answer!

Thank you for your answer, and another question is, I want to deblur faster for deployment in industrial (the performance can be degraded a little), what optimizations do you think can be done? Could you give some advice?

codeslake commented 2 years ago

Hi, @hust-lidelong. Please refer to this answer!

Thank you for your answer, and another question is, I want to deblur faster for deployment in industrial (the performance can be degraded a little), what optimizations do you think can be done? Could you give some advice?

I'm not an expert in the area. I would suggest searching with keywords, "network compression" or "network pruning" I guess.

hust-lidelong commented 2 years ago

Hi, @hust-lidelong. Please refer to this answer!

Thank you for your answer, and another question is, I want to deblur faster for deployment in industrial (the performance can be degraded a little), what optimizations do you think can be done? Could you give some advice?

I'm not an expert in the area. I would suggest searching with keywords, "network compression" or "network pruning" I guess.

If I just modify your network IFAN, I need the test time to be faster, Could you give some advice? Maybe slightly delete some layers?

codeslake commented 2 years ago

Hi, @hust-lidelong. Please refer to this answer!

Thank you for your answer, and another question is, I want to deblur faster for deployment in industrial (the performance can be degraded a little), what optimizations do you think can be done? Could you give some advice?

I'm not an expert in the area. I would suggest searching with keywords, "network compression" or "network pruning" I guess.

If I just modify your network IFAN, I need the test time to be faster, Could you give some advice? Maybe slightly delete some layers?

I tried running the RefVSR model from my recent paper using pytorch amp. You may try the code!

hust-lidelong commented 2 years ago

Hi, @hust-lidelong. Please refer to this answer!

Thank you for your answer, and another question is, I want to deblur faster for deployment in industrial (the performance can be degraded a little), what optimizations do you think can be done? Could you give some advice?

I'm not an expert in the area. I would suggest searching with keywords, "network compression" or "network pruning" I guess.

If I just modify your network IFAN, I need the test time to be faster, Could you give some advice? Maybe slightly delete some layers?

I tried running the RefvVSR model from my recent paper using pytorch amp. You may try the code! .

You mean torch.cuda.amp.GradScaler() can accelerate network inference time?

codeslake commented 2 years ago

Yes, inference time gets faster as most of the layers are to be operated in 16-bit precision.

hust-lidelong commented 2 years ago

Yes, inference time gets faster as most of the layers are to be operated in 16-bit precision.

Thanks for your welcome reply! And would you add this operator in the IFAN code?

codeslake commented 2 years ago

Thanks for your welcome reply! And would you add this operator in the IFAN code?

Hi, @hust-lidelong, I've updated the code. Please pull the repo.

Install PytTroch 1.11.0 and requirements (Note that to run PyTorch AMP, you need to have PyTorch > 1.10.0)
```
$ conda create -y --name IFAN python=3.8 && conda activate IFAN
$ sh install_CUDA11.3.sh
```

Train IFAN with PyTorch AMP

$ CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node=4 --master_port=9000 run.py \
        --is_train \
        --mode IFAN_amp \
        --config config_IFAN_amp \
        --trainer trainer \
        --network IFAN \
        -b 2 \
        -th 8 \
        -dl \
        -ss \
        -dist

Test IFAN with PyTorch AMP

$ CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN_amp --network IFAN --data DPDD --ckpt_sc

I am not sure about the performance drop, but I can see a 40% speed boost.

hust-lidelong commented 2 years ago

Wooo! So good and quick! Thanks very much!!! And I have one more question. In your paper, you present a reblurring network in fig.4, and "Note that we utilize the reblurring network only for training". I do not understand how it works and why you need to use it. Thanks！

codeslake commented 2 years ago

Wooo! So good and quick! Thanks very much!!! And I have one more question. In your paper, you present a reblurring network in fig.4, and "Note that we utilize the reblurring network only for training". I do not understand how it works and why you need to use it. Thanks！

The reblurring task is to regularize the predicted deblurring kernel F to have more valid per-pixel defocus blur information. For example, with the reblurring task, we can give the per-pixel size and shape of defocus blur. Refer to paragraphs around Eq. 3 for more detail, please!

hust-lidelong commented 2 years ago

Wooo! So good and quick! Thanks very much!!! And I have one more question. In your paper, you present a reblurring network in fig.4, and "Note that we utilize the reblurring network only for training". I do not understand how it works and why you need to use it. Thanks！

The reblurring task is to regularize the predicted deblurring kernel F to have more valid per-pixel defocus blur information. For example, with the reblurring task, we can give the per-pixel size and shape of defocus blur. Refer to paragraphs around Eq. 3 for more detail, please!

Thanks for your help! You are really nice!

hust-lidelong commented 2 years ago

Dear author： Your work is very nice. When I read your paper, I see Iterative Adaptive Convolution (IAC). I am a little confused about these two words iterative and adaptive. How IAC layer shows iterative and adaptive from two aspects. And by adopting iterative and adaptive , what is the benefit? Thanks!

codeslake commented 2 years ago

Dear author： Your work is very nice. When I read your paper, I see Iterative Adaptive Convolution (IAC). I am a little confused about these two words iterative and adaptive. How IAC layer shows iterative and adaptive from two aspects. And by adopting iterative and adaptive , what is the benefit? Thanks!

By adapively predicting deblurring filters given a defocused image, the network gains the flexibility in handling defocus blur, which has been shown in the ablation study.

By iteratively applying the predicted filter, we can establish a larger receptive field on the defocused features. This allows the better capability of the network in dealing with large defocus blur. The point is that IFAN iteratively applies separable filters to suppress the computational costs, which would have been intractable with larger 2-dim filters.

hust-lidelong commented 2 years ago

Dear author： Your work is very nice. When I read your paper, I see Iterative Adaptive Convolution (IAC). I am a little confused about these two words iterative and adaptive. How IAC layer shows iterative and adaptive from two aspects. And by adopting iterative and adaptive , what is the benefit? Thanks!

By adapively predicting deblurring filters given a defocused image, the network gains the flexibility in handling defocus blur, which has been shown in the ablation study.

By iteratively applying the predicted filter, we can establish a larger receptive field on the defocused features. This allows the better capability of the network in dealing with large defocus blur. The point is that IFAN iteratively applies separable filters to suppress the computational costs, which would have been intractable with larger 2-dim filters.

Thanks for your reply! I understand how it works.

codeslake / IFAN

How to train my own dataset #7