Unable to Reproduce Paper Results with Pretrained Models

[mvgshx]

Hi there,

Thank you for your great work!

I am facing issues reproducing the results for the REDS dataset as presented in the paper, using the pretrained models provided in this repository.

Environment

• Python Version: 3.8.19
• PyTorch Version: 1.10.1
• BasicSR Version: 1.3.5
• OS: Ubuntu 18.04.6 LTS
• GPU: Quadro RTX 6000

Steps to Reproduce

1. Downloaded the pretrained models from the provided Google Drive link, specifically spynet_sintel_final-3d2a1287.pth and SSL_REDS_BI.pt.
2. Set up the REDS dataset by following the Dataset Preparation instructions in the README.
3. Executed sh test_reds_BI.sh using the provided script.

Expected Results

According to the paper, I expected a PSNR/SSIM of 31.06/0.8933.

Actual Results

However, the results I obtained were significantly different, with a PSNR/SSIM of 27.9316/0.8338.

Additional Information

• No modifications were made to the codebase other than updating paths for the dataset and pretrained models in test_BasicVSR_REDS_BIx4.yml.
• I have ensured all dependency versions match those listed in the README.

I would appreciate any guidance on potential steps I might have missed or adjustments I could make to better align my results with those reported in the paper.

Thank you for your assistance and for sharing your work with the community!

[mvgshx]

Apologies, it seems that the REDS dataset was configured wrongly at my end. After a fresh download and regroup of the dataset, the model is able to reproduce the paper's results.