Single Image Example-based Super-Resolution

Description

Performs example-based Super-resolution on an image using direct mapping between high and low resolution patches.

Learning is done utilizing a self-example high-resolution, low-resolution coupled dictionary. Self-example means the dictionary is created from sampling patches from the input low resolution image. This approach is based on the observation that similar patches of a natural image occur within the same scale and between different scales.

An affine mapping approach is learned to transform a low resolution patch to its high resolution counterpart. This problem is ill-posed and hence we add a regularization term based on Tikhonov regularization.

We take patches of the image in the wavelet domain to perform training and prediction. We expand the discrete wavelet basis by using a discrete rational wavelet transform. Results suggest that the increased sparsity and the larger basis help to improve the ill-posed nature of the problem.

Super-resolution is performed through gradual upscaling. This method better retains low resolution patch structures and outperforms single step upscaling at higher upscaling rates.

API

gs::superResolution

cv::Mat* gs::superResolution(cv::Mat inputImage, float upscale, int iterations, double lambda, unsigned int patchSize unsigned int patchOverlap, unsigned int neighborhoodSize, double neighborhoodWeight);

gs::superResolution uses example patches from the input low resolution image to perform super-resolution.

Paramaters

cv::Mat inputImage: Input low resolution image.

float upscale: Upscaling factor. Must be greater than 1.

int iterations: Number of upscaling iterations performed. The final product is upscaled by a factor of 'upscale'.

double lambda: Regularization weight.

unsigned int patchSize: Patch size.

unsigned int patchOverlap: Patch overlap.

unsigned int neighborhoodSize: K-nearest neighbors used.

double neighborhoodWeight: K-nearest neighbor scaling factor. Dictionary atoms are scaled based on the atom's distance from a low resolution patch.

gs::superResolutionWavelet

cv::Mat* gs::superResolutionWavelet(cv::Mat inputImage, float upscale, int iterations, double lambda, unsigned int patchSize unsigned int patchOverlap, unsigned int neighborhoodSize, double neighborhoodWeight, int waveletP, int waveletQ);

gs::superResolutionWavelet uses example wavelet patches from the input low resolution image to perform super-resolution.

Paramaters

cv::Mat inputImage: Input low resolution image.

float upscale: Upscaling factor. Must be greater than 1.

int iterations: Number of upscaling iterations performed. The final product is upscaled by a factor of 'upscale'.

double lambda: Regularization weight.

unsigned int patchSize: Patch size.

unsigned int patchOverlap: Patch overlap.

unsigned int neighborhoodSize: K-nearest neighbors used.

double neighborhoodWeight: K-nearest neighbor scaling factor. Dictionary atoms are scaled based on the atom's distance from a low resolution patch.

int waveletP: upsampling wavelet factor.

int waveletQ: downsampling wavelet factor. WaveletQ must be larger than waveletP.

Example

#include "SuperResolution.h"
#define LR_IMAGE_PATH "../images/peppers256.tif"
#define GT_IMAGE_PATH "../images/peppers.png"

int main()
{

    cv::Mat imageLR;
    cv::Mat imageGT;

    /*load the test images*/
    cv::Mat imageLR = imread(LR_IMAGE_PATH, CV_LOAD_IMAGE_GRAYSCALE);
    imageLR = gs::to32F(imageLR);
    imageGT = imread(GT_IMAGE_PATH, CV_LOAD_IMAGE_GRAYSCALE);
    imageGT = gs::to32F(imageGT);

    /*SR variables*/
    float upscale = 2.0;
    int iterations = 1;
    unsigned int patchSize = 4;
    unsigned int patchOverlap = 3;
    double lambda = 1e-12;
    unsigned int neighborhoodSize = 200;
    double neighborhoodWeight = 1.0;

    /*Super-resolve the image*/
    Mat* imageSR = gs::superResolve(imageLR, upscale, iterations, lambda, patchSize, patchOverlap, neighborhoodSize, neighborhoodWeight);
    /*export the report and images*/
    gs::exportReport(imageLR, *imageSR, imageGT, upscale, iterations, lambda, patchSize, patchOverlap, neighborhoodSize, neighborhoodWeight);

    int waveletP = 7;
    int waveletQ = 8;
    neighborhoodWeight = 4.0;
    /*Super-resolve the image*/
    Mat* imageSRWavelet = gs::superResolveWavelet(imageLR, upscale, iterations, lambda, patchSize, patchOverlap, neighborhoodSize, neighborhoodWeight, waveletP, waveletQ);
    /*export the report and images*/
    gs::exportReportWavelet(imageLR, *s, imageGT, 2, 1, lambda, patchSize, patchOverlap, neighborhoodSize, neighborhoodWeight,waveletP,waveletQ);
}

Results

Input Image:

Super-resolved:

Bicubic Interpolation:

Ground Truth:

License

This project is licensed under the terms of the MIT license.

[1] Bevilacqua, Marco, et al. "Single-image super-resolution via linear mapping of interpolated self-examples." IEEE Transactions on Image Processing 23.12 (2014): 5334-5347.