search

clovaai / cutblur

Rethinking Data Augmentation for Image Super-resolution (CVPR 2020)
MIT License
382 stars 62 forks source link

Is the test result the average value of multiple models? #20

Open yelusaleng opened 3 years ago

yelusaleng commented 3 years ago

hi authors, i‘ve only tested the performance of cutblur once by using python main.py --model CARN --augs cutblur --alpha 0.7 --dataset RealSR --scale 4 --camera all --dataset_root ./input/RealSR/ --ckpt_root ./pt/RealSR/cutblur/ --save_result --save_root ./output/RealSR/cutblur/. the obtained result is 28.89 which is lower than the result 29.00 in the paper.

therefore, i would like to know if 29.00 is the average of multiple models tested.

nmhkahn commented 3 years ago

In the RealSR dataset, we have observed that early stopping is crucial for the best performance. The overfitting seems to worsen when the network is bigger (e.g. EDSR, RCAN) and when not using data augmentation, obviously. Is the 28.89 PSNR of the "best"? or "the last" result? By the way, all the performance reports are measured a single time.

yelusaleng commented 3 years ago

thank you for your response. the 28.89 is the best, not the last.