xinntao / Real-ESRGAN-ncnn-vulkan

NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.
https://github.com/xinntao/Real-ESRGAN
Other
1.46k stars 182 forks source link

Can not convert RealESRGAN output to OpenCV image in C++ #50

Closed Seprentes closed 1 year ago

Seprentes commented 1 year ago

I want to use RealESGRAN class at my project so i wrote this code for learning to use RealESGRAN class:

#include <cstddef>
#include <cstdio>
#include <iostream>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include "realesrgan.h"

int main() {
    int gpuid = 1, tilesize;
    uint32_t heap_budget = ncnn::get_gpu_device(gpuid)->get_heap_budget();

    if (heap_budget > 1900)
        tilesize = 200;
    else if (heap_budget > 550)
        tilesize = 100;
    else if (heap_budget > 190)
        tilesize = 64;
    else
        tilesize = 32;

    RealESRGAN upscaler = RealESRGAN(gpuid, true);

    cv::Mat img = cv::imread("corlu.jpg");
    int w = img.cols;
    int h = img.rows;

    int scale = 4;

    ncnn::Mat in = ncnn::Mat::from_pixels((unsigned char *)img.data, ncnn::Mat::PIXEL_BGR, w, h);
    cv::Mat output_img = cv::Mat::zeros(h * scale, w * scale, CV_8UC3);
    ncnn::Mat out = ncnn::Mat::from_pixels((unsigned char *)output_img.data, ncnn::Mat::PIXEL_BGR, w * scale, h * scale);
    //ncnn::Mat out = ncnn::Mat(w * scale, h * scale, (size_t)3, 3);
;

    upscaler.load("models/realesrgan-x4plus.param", "models/realesrgan-x4plus.bin");

    upscaler.scale = scale;
    upscaler.tilesize = tilesize;
    upscaler.prepadding = 10;

    upscaler.process(in, out);

    out.to_pixels(output_img.data, ncnn::Mat::PIXEL_BGR);
    cv::imwrite("output.jpg", output_img);
}

If i use code without changes i get black image with some blue things and if i useout = ncnn::Mat(w * scale, h * scale, (size_t)3, 3); as out i get segmentation fault at out.to_pixels(output_img.data, ncnn::Mat::PIXEL_BGR); part of my code.

Seprentes commented 1 year ago

I solved the issue with this code:

#include <cstddef>
#include <cstdio>
#include <iostream>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include "realesrgan.h"

int main() {
    int gpuid = 1, tilesize;
    uint32_t heap_budget = ncnn::get_gpu_device(gpuid)->get_heap_budget();

    if (heap_budget > 1900)
        tilesize = 200;
    else if (heap_budget > 550)
        tilesize = 100;
    else if (heap_budget > 190)
        tilesize = 64;
    else
        tilesize = 32;

    RealESRGAN upscaler = RealESRGAN(gpuid);

    cv::Mat img = cv::imread("corlu.jpg");

    int w = img.cols;
    int h = img.rows;

    int scale = 4;

    ncnn::Mat in = ncnn::Mat(w, h, (void *)img.data, (size_t)3, 3);
    ncnn::Mat out = ncnn::Mat(w * scale, h * scale, (size_t)3, 3);

    upscaler.load("models/realesrgan-x4plus.param", "models/realesrgan-x4plus.bin");

    upscaler.scale = scale;
    upscaler.tilesize = tilesize;
    upscaler.prepadding = 10;

    upscaler.process(in, out);

    cv::Mat output_img = cv::Mat(out.h, out.w, CV_8UC3, out.data);
    cv::imwrite("output.jpg", output_img);
}