Single Image Super-Resolution via CNN Architectures and TV-TV Minimization
Introduction
Matlab code that replicates the experiments in the paper:
Single Image Super-Resolution via CNN Architectures and TV-TV Minimization
Marija Vella, João F. C. Mota
BMVC 2019
The paper describes a post-processing step for any super-resolution algorithm, but which is particularly relevant for CNN-based algorithms.
Given a low-resolution image b and the output w of a super-resolution algorithm, the post-processing step produces an improved high-resolution image by solving TV-TV minimization:
Our experiments show that this procedure step systematically improves the quality of the reconstructed image, as measured by PSNR and SSIM, even when the base algorithms are state-of-the-art, (e.g., Kim, SRCNN, SelfExSR, DRCN).
Requirements
- Matlab (tested for R2019a)
- Image processing toolbox
Contents
There are 2 main folders: images and code.
The images folder contains three subfolders:
-
Test_Sets_GT - Ground truth images from the datasets Set5, Set14, and BSD100; these are used for testing.
-
Test_Sets_Side - Output images of the super-resolution methods we considered (Kim, SRCNN, SelfExSR and DRCN). These were cropped to the appropriate size according to the scaling factor to avoid misalignment with the ground truth images.
-
SampleImages - Contains two subfolders with sample images, one with the ground truth images and another with output images from SRCNN for an upscaling factor of 2x.
The code folder contains the code required to run all the experiments. The script experiments.m executes a simple test; but if some lines are uncommented, all experiments in the paper are performed. The folder also contains two implementations of our solver, one optimized for GPUs and another for CPUs, and additional auxiliary code.
Comparison with state-of-the-art Single-Image Super-Resolution methods
Testing Datasets
The ground truth images used for all experiments were obtained here, and can also be found in the following links
* Set5 Dataset
-
Testing Datasets used as side information
We used the following super-resolution methods for comparison and as base algorithms:
| Method | Citation |
|---|---|
| Kim | K. I. Kim and Y. Kwon, “Single-image super-resolution using sparse regression and natural image prior”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 6, pp. 1127-1133, 2010. Code available here. |
| SRCNN | C.Dong, C.C. Loy, K. He, X. Tang "Learning a Deep Convolutional Network for Image Super-Resolution, ECCV 2014. Code available here. |
| SelfExSR | J.Huang, A. Singh and N. Ahuja, "Single Image Super-Resolution from Transformed Self-Exemplars", CVPR 2015. Code available here. |
| DRCN | J. Kim, J. Kwon Lee and K.M. Lee, " Deeply-Recursive Convolutional Network for Image Super-Resolution", CVPR 2016. Code available here. |
For Kim, SRCNN and SelfExSR, we used output images available here. For DRCN, we used the output images made available by the authors themselves. The super-resolved images of the respective methods used as side information (w) in our method can be found in the following links:
Quantitative Results
The values in the table represent PSNR and SSIM (in parenthesis), and
the larger their value the better.
