[Bug]: FakeQuant failed during import_model(exported quantized_compiled_model) when F.interpolate or nn.Upsample exists

OpenVINO Version

2024.3.0

Operating System

Windows System

Device used for inference

CPU

Framework

PyTorch

Model used

Resnet50, UNet

Issue description

FakeQuant Failed during import_model(exported nncf.quantized_compiled_model) when F.interpolate or nn.Unsample exists

byte_data = compiled_int8_model.export_model()
core = ov.Core()
new_compiled = core.import_model(io.BytesIO(byte_data), "CPU")

Traceback (most recent call last):

  File "D:\project\code\z_dev\fake_sseg.py", line 199, in <module>
    new_compiled = core.import_model(io.BytesIO(byte_data), "CPU") # Will Failed
  File "D:\PythonEnv\py39\lib\site-packages\openvino\runtime\ie_api.py", line 602, in import_model
    super().import_model(
RuntimeError: Exception from src\inference\src\cpp\core.cpp:187:
Exception from src\inference\src\dev\plugin.cpp:73:
Check 'input_shapes.size() == 2' failed at src\core\shape_inference\include\interpolate_shape_inference.hpp:243:
While validating node 'opset1::Interpolate Interpolate_13911 (opset1::FakeQuantize __module.encoder/aten::_convolution/Add/fq_output_0[0]:f32[1,64,254,254], opset1::Constant Constant_12126[0]:i32[2], opset1::Constant __module.upsample/aten::upsample_nearest2d/Multiply[0]:f32[2], opset1::Constant Constant_59[0]:i32[2]) -> ()' with friendly_name 'Interpolate_13911'

Step-by-step reproduction

Code

import torch, io, nncf
import torch.nn as nn
import torch.nn.functional as F
import openvino as ov
import numpy as np

W, H = 256, 256

class simpleUpsample(nn.Module):
    def __init__(self):
        super().__init__()
        self.encoder = nn.Conv2d(3, 64, 3, 1)
        self.upsample = nn.Upsample(scale_factor=2, mode='nearest')
        # self.decoder = nn.Conv2d(64, 2, 3, 1)

    def forward(self, x):
        # return self.decoder(self.upsample(self.encoder(x)))
        return self.upsample(self.encoder(x))

class SSegDataset(torch.utils.data.Dataset):
    def __len__(self):
        return 100

    def __getitem__(self, idx):
        image = np.random.randn(3, H, W).astype(np.float32)
        return image, 0

model = simpleUpsample()
model.eval().cpu()
dummy_input = torch.randn([1, 3, H, W], device='cpu', dtype=torch.float32)

sseg_ds = SSegDataset()
clf_loader = torch.utils.data.DataLoader(sseg_ds, batch_size=1, shuffle=True)

def transform_fn(data_item):
    images, _ = data_item
    return images.numpy()
calibration_dataset = nncf.Dataset(clf_loader, transform_fn)

ir_model = ov.convert_model(model, example_input=dummy_input, input=[1, 3, H, W], verbose=True)
q_model = nncf.quantize(ir_model, calibration_dataset, target_device=nncf.TargetDevice.CPU, subset_size=50, ignored_scope=None)
c_model = ov.compile_model(q_model, device_name="CPU", config=None)

img = np.random.randn(1, 3, H, W).astype(np.float32)
result = c_model([img])[c_model.output(0)]
print(f"compile_model testing Done. result.shape: {result[0].shape}")

byte_data = c_model.export_model()
core = ov.Core()
new_compiled = core.import_model(io.BytesIO(byte_data), "CPU")
result = new_compiled([img])[new_compiled.output(0)]
print(f"export then import testing Done. result.shape: {result[0].shape}")

Environment OS windows 11 python 3.9; pytorch-2.3.1; openvino-2024.3.0; nncf-2.12.0

Relevant log output

Model Conversion arguments:
Common parameters:
        - Input Model:  <class 'openvino.frontend.pytorch.ts_decoder.TorchScriptPythonDecoder'>
        - Input layers:         [1, 3, 256, 256]
        - Output layers:        Not specified, inherited from the model
[ SUCCESS ] Total execution time: 6.38 seconds. 
[ SUCCESS ] Peak memory consumption (includes only memory allocated in Python): 77.73 MB. 
Statistics collection ━━━━━━━━━━━━━━━━ 100% 50/50 • 0:00:03 • 0:00:00
Applying Fast Bias correction ━━━━━━━━━━ ━ 100% 1/1 • 0:00:00 • 0:00:00
compile_model testing Done. result.shape: (64, 508, 508)
Backend tkagg is interactive backend. Turning interactive mode on.
Traceback (most recent call last):
  File "d:\project\code\pyjvl2\z_dev\fake_sseg.py", line 191, in <module>
    new_compiled = core.import_model(io.BytesIO(byte_data), "CPU")
  File "D:\PythonEnv\py39\lib\site-packages\openvino\runtime\ie_api.py", line 602, in import_model
    super().import_model(
RuntimeError: Exception from src\inference\src\cpp\core.cpp:187:
Exception from src\inference\src\dev\plugin.cpp:73:
Check 'input_shapes.size() == 2' failed at src\core\shape_inference\include\interpolate_shape_inference.hpp:243:
While validating node 'opset1::Interpolate Interpolate_13911 (opset1::FakeQuantize __module.encoder/aten::_convolution/Add/fq_output_0[0]:f32[1,64,254,254], opset1::Constant Constant_12126[0]:i32[2], opset1::Constant __module.upsample/aten::upsample_nearest2d/Multiply[0]:f32[2], opset1::Constant Constant_59[0]:i32[2]) -> ()' with friendly_name 'Interpolate_13911'

Issue submission checklist

[X] I'm reporting an issue. It's not a question.
[X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
[X] There is reproducer code and related data files such as images, videos, models, etc.

openvinotoolkit / openvino

[Bug]: FakeQuant failed during import_model(exported quantized_compiled_model) when F.interpolate or nn.Upsample exists #26111