SUPERRESOLUTIONS IR NOT WORKING

[fnando1995]

Hello,

I'm trying to use the superresolution model. But when running the infer_ie.py file, the final image is a big black rectangle.

Also this warning appears

/usr/local/lib/python3.5/dist-packages/skimage/util/dtype.py:135: UserWarning: Possible precision loss when converting from float32 to uint8

nothing from the file has being changed except a sorted in load_ir_model() because i check that the main uses de inputs in order.

SO: ubuntu 16.04.04 python version: 3.5.2 scikit-image==0.15.0 opencv-openvino: 4.1.0-openvino

[AlexanderDokuchaev]

Hi @fnando1995 /usr/local/lib/python3.5/dist-packages/skimage/util/dtype.py:135: UserWarning: Possible precision loss when converting from float32 to uint8 is standard warning when converting float32 to uint8 in skimage module, it's ok.

What model did you use? If you use model from model_zoo try this demo.

[fnando1995]

@AlexanderDokuchaev Thanks for your answer. I use the model "super-image-super-resolution-1021" not the 1033 as mention in the link.

this is my code. just change the argsparser. the input image size as the input of the network. and the cubic-image too.

from argparse import ArgumentParser
import os
import cv2
import skimage
import numpy as np
from openvino.inference_engine import IENetwork, IEPlugin
def load_ir_model(model_xml, device):
    model_bin = os.path.splitext(model_xml)[0] + ".bin"
    # initialize plugin and read IR
    plugin = IEPlugin(device=device)
    net = IENetwork(model=model_xml, weights=model_bin)
    exec_net = plugin.load(network=net)
    input_blobs = sorted(net.inputs.keys())
    inputs = [(b, net.inputs[b].shape) for b in input_blobs]
    out_blob = next(iter(net.outputs))
    del net
    return exec_net, plugin, inputs, out_blob
def image_to_blob(image, shape):
    blob = image.copy()
    blob = skimage.img_as_float32(blob)
    blob = blob.transpose((2, 0, 1))  # from HWC to CHW
    blob = blob.reshape(shape)
    return blob
def blob_to_img(blob):
    blob = blob.transpose((1, 2, 0))   # from CHW to HWC
    blob = np.clip(blob, 0.0, 1.0)
    blob = skimage.img_as_ubyte(blob)
    return blob
def main():
    MODEL="models/single-image-super-resolution-1021.xml"
    DEVICE="CPU"
    INPUT_IMAGE="img1.jpg"
    exec_net, _, inputs, out_blob = load_ir_model(MODEL,DEVICE)
    ih, iw = inputs[0][1][2:]
    image = cv2.resize(cv2.imread(INPUT_IMAGE),(iw,ih))
    cubic = cv2.resize(image, tuple(inputs[1][1][2:]), interpolation=cv2.INTER_CUBIC)  
    blob1 = image_to_blob(image, (inputs[0][1]))
    blob2 = image_to_blob(cubic, (inputs[1][1]))
    # inference
    result = exec_net.infer(inputs={inputs[0][0]: blob1, inputs[1][0]: blob2})
    out_img = blob_to_img(result[out_blob][0])
    out_path = os.path.join(os.path.dirname(INPUT_IMAGE), "sr_" + os.path.basename(INPUT_IMAGE))
    cv2.imwrite(out_path, out_img)
    print("Saved: ", out_path)
if __name__ == "__main__":
    main()

[AlexanderDokuchaev]

@fnando1995 So, found problem. In model_zoo located model that was converted with option scale_values. Remove line blob = skimage.img_as_float32(blob) from your code.

And found bug in demo: cubic = cv2.resize(image, tuple(inputs[1][1][2:]), interpolation=cv2.INTER_CUBIC) should be replaced to: cubic = cv2.resize(image, (inputs[1][1][3], inputs[1][1][2]), interpolation=cv2.INTER_CUBIC)

[fnando1995]

@AlexanderDokuchaev That makes the trick! thanks. But got other problem. Maybe i am not checking well, but the CUBIR_RESIZED_IMAGE and the SUPER_RESO_IMAGE are kind of the same... and also, 4x scale maybe not need. What i want to accomplish with super_reso, is have a bigger image (1.5x or 2.x) without blur, because i will use OCR after it.

Is this the correct way to do? or maybe some other will be better?

thanks!

[AlexanderDokuchaev]

Hm, you can modify model, remove branch for cubic image, and use only one PixelShuffle layer with factor=2. And train the model using your dataset. But, in my mind, better look on 'deblurring` models.

[fnando1995]

@AlexanderDokuchaev Thanks for your advice. Since the problem was solved, i will close this issue.