Patch Modification

[JadenLy]

Hi,

I am interested in modifying the patch implementation here. I have a csv file that records all box location for my training images. And I only want these boxes to be the patch for training. I have found that I need to add an argument in option.py for my csv file and modify get_patch in common.pyto return the box coordinates. In addition, insrdata.py, I need to add another attribute for my csv file, also modifyget_patchto add one more parameter to callget_patchfromcommon.py`. What did I miss here? I wonder how you implemented the number of patches for each image, since the number of boxes for each image may be different.

Thanks for your help !!

[sanghyun-son]

Hello.

I think you have to modify some codes.

First, a _scan function.

Please re-define the _scan function so that it can read the filename and box information from a single csv file.

Then, type these codes after calling _scan.

self.images_hr = list_hr
self.images_hr = list_lr

Then, fix these lines so that your code can directly read a image and crop a patch from the image.

After modification, use an additional --ext img argument to execute the code.

I think this approach will work, but please let me know if you have any problem.

Thank you.

[JadenLy]

Hello,

Thanks for your reply. I have some questions first.

1. I can rename all my data files in the same pattern as DIV2K, is it still necessary for me to modify _scan and _load_file?

2. Based on the point above, I was thinking that I only need to modify the get_patch in common.py. So I wonder if the idx parameter in PyTorch starts with 0 or 1?

Thanks~

[JadenLy]

Also, I have trained an RDN model. But when I use this trained model to run through images, I only get some gray images (I guess they are residual images). So how can I get the actual output?

I train the model using

python main.py --scale 2 --save RDN_D16C8G64_BIx2 --model RDN --epochs 200 --batch_size 128 --data_range 1-54497/54498-57366 --patch_size 8 --ext sep --dir_data ~/data/ --save_models

And I run the demo using

python main.py --data_test Demo --scale 2 --pre_train ~/EDSR-PyTorch/experiment/RDN_D16C8G64_BIx2/model/model_best.pt --patch_size 8 --chop --test_only --save_results

[sanghyun-son]

Hello.

If you can, it is possible to keep _scan and _load_file, and PyTorch index starts from 0.

For the second comment, please use --save_results option during the training phase to check whether your model is trained appropriately or not.

Please let me know if you require additional help!

Thank you.

[JadenLy]

Hello,

Thanks for your advice. I have completed most of the editing except the resizing. I notice that the image lr and hr in get_patch function are in the class imageio.core.util.image. I wonder how I can resize it to a specific dimension. Say, I want to resize it to 400x400. What is the way to do this. I found that PIL package is much easier, but it seems pytorch does not recognize it.

Thanks

[sanghyun-son]

You can use imresize function in scipy.misc for resizeing.

Since lr and hr are just numpy arrays, you can also use any resizing functions that support numpy.

If you prefer PIL, please convert the numpy arrays to PIL image first and then process.

Thank you!

[JadenLy]

@thstkdgus35

I have decided to take an alternate path to generate about 630,000 images and use the whole images as input. This way I can just ignore the patch mechanism. However, I found that when I use this dataset to train the model, it does not show any training output and directly goes into evaluation phase. If I lower the training and validation set to about 500 images and 100 images, the training process is normal. I used the following command

python main.py --scale 2 --save RDN_patch --model RDN --epochs 200 --batch_size 16 --data_range 1-600000/600001-630000 --patch_size 100 --ext sep --dir_data ~/new/ --save_results --chop --save_models

Then the output is

Making model... Preparing loss function: 1.000 * L1 [Epoch 1] Learning rate: 1.00e-4

Evaluation: 0%| | 9/30000 [00:04<3:51:50, 2.16it/s]

By the way, I only modify the code in get_patch function in common.py. I tried to debug by printing the length of data loader in trainer.py. When I use over 10000 images, it always return 0; if I only use thousands of images, if returns some positive number, but still weird. For example, I use 600 training images, it returns something like 10XX; when I use 6000 images, it returns 750. I wonder what is wrong here.

Do you know what could be wrong here? Thanks

[sanghyun-son]

@JadenLy Sorry for late response.

Please modify these lines.

I think setting self.repeat to 1 will solve your problem.

Thank you.

[JadenLy]

Thanks for your help. My training can successfully run now.

However, there seem to be a problem. While each epoch takes, based on the number in log, less than an hour, it seems that it takes an extremely long time to process between two epochs. I checked the log and noticed that only five epochs have been finished in about 15 hours. I wonder what other factors can influence here. Thanks you.

[sanghyun-son]

Hello.

I think the bottleneck is patch processing.

Did you check the data time and model time?

In the log, you can see the time information like time: ## + ##.

The previous one is the model time, and the other is the data time.

You may pre-process the patches to reduce the data time if it was the bottleneck.

Thank you.