[Bug]: OpenVINO and PyTorch gave an different output shape for avg_pool2d

jikechao commented 10 months ago

OpenVINO Version

2023.1.0-12185-9e6b00e51cd-releases/2023/1

Operating System

Ubuntu 18.04 (LTS)

Device used for inference

CPU/GPU

Framework

PyTorch

Model used

No response

Issue description

For the model shown in the below script, OpenVINO and PyTorch gave an different output shape.

Step-by-step reproduction

import torch
from torch.nn import Module
import openvino as ov
import numpy as np

def compile_torch(model, input_data):
    ov_model = ov.convert_model(model, example_input=input_data)
    ir_path = f"temp_OVIR.xml"
    ov.save_model(ov_model, ir_path)
    core = ov.Core()
    model = core.read_model(ir_path)

    compiled_model = core.compile_model(model=model, device_name='CPU')
    output_key = compiled_model.output(0)
    result = compiled_model(input_data)[output_key]
    return result

input_data = torch.randn([1, 16, 4, 4], dtype=torch.float32)

class avg_pool2d(Module):
    def forward(self, *args):
        return torch.nn.functional.avg_pool2d(args[0], ceil_mode=True,  kernel_size=(1, 2), padding=(0, 1), stride=2)

torch_model = avg_pool2d().float().eval()
torch_outputs = torch_model(input_data).cpu().numpy()

trace = torch.jit.trace(torch_model, input_data)
trace = torch.jit.freeze(trace)

input_shapes = input_data.shape
res_ov = compile_torch(trace, input_data)
np.testing.assert_allclose(torch_outputs, res_ov, rtol=1e-3, atol=1e-3)

Relevant log output

AssertionError: 
Not equal to tolerance rtol=0.001, atol=0.001

(shapes (1, 16, 2, 3), (1, 16, 3, 3) mismatch)
 x: array([[[[-0.018069,  0.675706,  0.018802],
         [-0.62903 ,  0.265722, -0.033519]],
...
 y: array([[[[-0.018069,  0.675706,  0.018802],
         [-0.62903 ,  0.265722, -0.033519],
         [ 0.      ,  0.      ,  0.      ]],...

Issue submission checklist

[X] I'm reporting an issue. It's not a question.
[X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
[X] There is reproducer code and related data files such as images, videos, models, etc.

ilya-lavrenov commented 10 months ago

Hi @jikechao Is it GPU specific issue? How does it work on CPU? It will help to determine a component to investigate

jikechao commented 10 months ago

@ilya-lavrenov GPU and CPU and the same problem. Both of them in OpenVINO have different output shapes with PyTorch.

mvafin commented 10 months ago

That is a known issue. Same as for MaxPool, however it was fixed for MaxPool https://github.com/openvinotoolkit/openvino/blob/9decbb538bcd791cbc394cd18f8754370e1b300a/src/frontends/pytorch/src/op/max_poolnd.cpp#L58-L101 But the fix is not great in terms of performance and it generates incorrect indices for that case. This problem is caused by a known issue in torch, which wouldn't be fixed AFIK and can be treated as a feature at this point. We are discussing a possibility to support it with a new opset version of pooling ops.

openvinotoolkit / openvino