search

karlygzhu / RGAN

5 stars 0 forks source link

Try to apply this model to blind image #2

Open Explore008 opened 8 months ago

Explore008 commented 8 months ago

Hello! I created a demo file according to your test, intending to use the user's own pictures to do super resolution test, but there are some problems:

  1. Your original model is to perform Gaussian downsampling and then restore the picture. If you cancel the downsampling process and directly input a picture for testing, the quality will be somewhat improved, but it will still be very fuzzy, and the effect will not be as good as that of downsampling and then recovery (including using the picture in the data set).
  2. When I visualized your subsampling LRtensor, I found that it was reasonable from the perspective of dimensional change, but the output image was a completely black image, I don't know whether there were some problems in the process of subsampling
  3. When testing the output, the picture produced some white striped things, may I ask why Thank you very much! source_0 model_0 source_1 model_1
karlygzhu commented 8 months ago

1、Our model is a non-blind video super-resolution (VSR) model. The effect may be somewhat reduced when performing blind VSR. In addition, if you simply eliminate the downsampling step and input the already downsampled LR frames directly into the model, the results of the refactoring will not change. If you enter real high-resolution frames, you can modulate the generated frames to be the same size as original frames. Generally, the generated frames will have better visualization. 2、When converting the read data type to tensor, I normalize the data. The code is “HR_all = [trans_tensor(HR) for HR in HR_all]”. I presume that when you output the LR frame image, you don't restore the image, which results in an output that is always black. You need to recover and store data, using the function save_img in the test.py file. In the save_img function, the fourth argument att is False. 3、This might be that when the model is performing long-distance spatio-temporal information transfer, the information at a longer distance adversely affects the generation of the current frame. You can try shortening the length of video for each training session and observe if it can defuse the problem.