search

ckkelvinchan / RealBasicVSR

Official repository of "Investigating Tradeoffs in Real-World Video Super-Resolution"
Apache License 2.0
906 stars 136 forks source link

About the evaluation #26

Closed CrissyHoo closed 2 years ago

CrissyHoo commented 2 years ago

I used the official NIQE code to evaluate the demo_000 and the result, got a unexpected result, as the niqe value of the raw video is 3.9829 while the sr video is 4.3407. I just input every frame and calculate the average value. I don't know where is wrong, as this result is totally opposite towards that in paper.

ckkelvinchan commented 2 years ago

Different resolutions could lead to different NIQE. Therefore we upsample the input to maintain the same resolution during comparison. You can apply 4x upsampling first and test the NIQE.

CrissyHoo commented 2 years ago

Thanks for the fast response! but there are lots of different upsampling ways, could you tell me which way is used in this paper?

ckkelvinchan commented 2 years ago

I used the default imresize in MATLAB. That means bicubic downsampling.

CrissyHoo commented 2 years ago

I used the default imresize in MATLAB. That means bicubic downsampling.

hello～I have another question during testing on videolq dataset. Could you tell me the devices(how many and which type of GPUs are used when testing)? Cause I noticed that the vid4 dataset in videolq is usually used as a high quality files in most sr methods but in this paper this is regarded as the low quality one. This will lead to an increasing computation when testing.

ckkelvinchan commented 2 years ago

I use one V100 GPU (32GB) for test.

CrissyHoo commented 2 years ago

I use one V100 GPU (32GB) for test.

I noticed that you use a parameter "max_seq_len" when testing, could you tell me what's your choice? Cause I just input the whole frames of one video and then i got the error cuda out of memory with one v100 gpu. Is that the reason that you set this hyper-param?

ckkelvinchan commented 2 years ago

The results provided in the paper use the entire sequence. The max_seq_len is used for sequences that are much longer than 100 frames.