openvinotoolkit / model_api

https://openvinotoolkit.github.io/model_api/latest
Apache License 2.0
26 stars 16 forks source link

Fix `resize_image_ocv` for `fit_to_window` #169

Closed sungchul2 closed 7 months ago

sungchul2 commented 8 months ago

What does this PR do?

Fixes https://github.com/openvinotoolkit/model_api/issues/167

Example source:

import torch
import tempfile
import numpy as np
import matplotlib.pyplot as plt
import openvino
from openvino.model_api.adapters import OpenvinoAdapter, create_core
from openvino.model_api.adapters.utils import resize_image_with_aspect_ocv
from openvino.preprocess import PrePostProcessor

class DummyModel(torch.nn.Module):
    def forward(self, inputs):
        return inputs

model = DummyModel()

with tempfile.TemporaryDirectory() as tmpdirname:
    onnx_path = tmpdirname + "/dummy.onnx"
    ov_path = tmpdirname + "/dummy.xml"
    torch.onnx.export(
        model,
        (torch.zeros(1, 3, 1024, 1024, dtype=torch.float32)),
        open(onnx_path, "wb"),
        input_names=["inputs"],
        output_names=["outputs"])
    ov_model = openvino.convert_model(onnx_path)
    openvino.save_model(ov_model, ov_path, compress_to_fp16=False)

    model_adapter = OpenvinoAdapter(
        core=create_core(),
        model=ov_path,
        weights_path=ov_path.replace(".xml", ".bin"),
        device="CPU",
        max_num_requests=1,
        plugin_config={"PERFORMANCE_HINT": "THROUGHPUT"},
    )

    model_adapter.embed_preprocessing(
        layout="NCHW",
        resize_mode="fit_to_window",
        interpolation_mode="LINEAR",
        target_shape=(1024, 1024),
        pad_value=0,
        brg2rgb=False,
    )

img = np.ones((256, 512, 3)).astype(np.uint8) * 255
ov_results = model_adapter.compiled_model(img[None])
np_results = resize_image_with_aspect_ocv(img, (1024, 1024))

plt.figure(figsize=(15,7))
plt.subplot(131)
plt.imshow(img)
plt.title("input image w/ (256, 512)")
plt.subplot(132)
plt.imshow(np_results)
plt.title(f"resize_image_with_aspect_ocv results w/ {np_results.shape[:2]}")
plt.subplot(133)
plt.imshow(list(ov_results.values())[0][0].transpose(1, 2, 0))
plt.title("openvino results w/ (1024, 1024)")
plt.show()

Before: image

After: image

Before submitting

sungchul2 commented 7 months ago

@sovrasov @Wovchena Could you take a look at failed test cases? They occurred during compiling models.

sovrasov commented 7 months ago

@sungchul2 thanks, the PR looks good. Could you add the reproducer from the PR description to tests/python/precommit/ as a unit test? I believe we could check that sum(abs(ov_image - np_image)) < error