search

pytorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration
https://pytorch.org
Other
83.74k stars 22.58k forks source link

Parameterless model still has extra inputs when exported to ONNX + insufficient checking of argument export_params #55144

Open vadimkantorov opened 3 years ago

vadimkantorov commented 3 years ago

Note the wrong setting of export_params = None - probably should be checked by PyTorch to only be True or False (if None is accepted, its semantics should be clearly stated in the docs). Somehow the exported model has three inputs: one tensor and two int64. Where are they coming from?

image

Original report: https://github.com/microsoft/onnxruntime/issues/7096 

import onnxruntime
from torch import nn
import torch
import torch.nn.functional as F

print(torch.__version__)

class Model(nn.Module):
    def forward(self, x):
        p = F.pad(x, (1, 1), mode='constant').squeeze(1)
        return dict(o=p)

model = Model()
model.to(device='cuda')
x = torch.rand(1, 10).to(device='cuda')
print(x.shape)
print(model(x))

torch.onnx.export(
        model, (x,),
        'fp32_squeeze_repro.onnx',
        verbose=False,
        opset_version=12,
        export_params=None,
        do_constant_folding=True,
        input_names=['x'],
        output_names = ['o']
)

runtime = onnxruntime.InferenceSession('fp32_squeeze_repro.onnx')
print(runtime.run(None, dict(x=x.cpu().numpy())))

#1.8.0
#torch.Size([1, 10])
#{'o': tensor([[0.0000, 0.7245, 0.8421, 0.7092, 0.6501, 0.6383, 0.4523, 0.5001, 0.9790,
#         0.5590, 0.3269, 0.0000]], device='cuda:0')}
#Traceback (most recent call last):
#  File "fp32_squeeze_repro.py", line 33, in <module>
#    print(runtime.run(None, dict(x=x.cpu().numpy())))
#  File "/opt/conda/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 120, in run
#    raise ValueError("Model requires {} inputs. Input Feed contains {}".format(num_required_inputs, num_inputs))
#ValueError: Model requires 3 inputs. Input Feed contains 1

fp32_squeeze_repro.onnx.zip

cc @houseroad @spandantiwari @lara-hdr @BowenBao @neginraoof @SplitInfinity

jiafatom commented 3 years ago

This is being tracked internally by Microsoft at https://msdata.visualstudio.com/Vienna/_workitems/edit/1446551

@vadimkantorov is there any reason that you export through export_params=None?

jiafatom commented 3 years ago

For squeeze(1) here, actually p = F.pad(x, (1, 1), mode='constant'). returns the tensor shape [1, 12]. pytorch squeeze(1) will do nothing because 12!=1. However, onnx spec does not allow this. It needs the shape entry of Squeeze axis be 1.

For your case here, we don't need squeeze(1). Then with export_params=True, the model can work.

For general case to support this squeeze issue, I filed an issue in onnx repo: https://github.com/onnx/onnx/issues/3764

vadimkantorov commented 3 years ago

export_params=None was by mistake. I was reporting that it should probably be enforced of being only True or False, otherwise it should be documented that None is also an allowed default value

jiafatom commented 3 years ago

It is documented here: https://github.com/pytorch/pytorch/blob/201174cb91d7b61798bf543cf001dcb68fadbf50/torch/onnx/__init__.py#L103-L106 It says export_params (bool, default True) when you call torch.onnx.export.__doc__, you will see it

vadimkantorov commented 3 years ago

According to docs, non-bool value (such as None) should trigger an error, but this is not the case.

If None has some special interpretation, e.g. it would reset to the default value of True (reset to default value is a frequent function of None), it should be clearly stated.

My complain is that I passed None by mistake and it was not rejected. It may be fine that it converts any thing to bool by type casting bool(export_params), but maybe it should be clearly stated.

vadimkantorov commented 3 years ago

Also, in the particular example in my original complaint, there appeared additional two inputs while the original model had only one input. That model had no parameters at all as well.

jiafatom commented 3 years ago

(1) User experience. Agree that we need have better user experience on this, tracked internally by Microsoft at https://msdata.visualstudio.com/Vienna/_workitems/edit/1447625 (2) Additional two inputs. As mentioned in previous comments, Tracked internally by Microsoft at https://msdata.visualstudio.com/Vienna/_workitems/edit/1446551 This is because the constant folding pass does not handle well for some export_params=False/None case. For export_params=True, there is no such issues. (3) Even if you try export_params=True, this particular model has issue for squeeze(1), as explained above, that is the onnx spec mismatches pytorch squeeze. Have filed a feature request for onnx repo as above.

vadimkantorov commented 3 years ago

Thank you!

thiagocrepaldi commented 1 year ago

For the last item, although if-branch mismtaches else-branch, when dynamic_axes is specified to the exporter, ORT's InferenceSession can execute it with a warning, but when it is commented out, inference session on the ONNX model fails

import numpy as np
import onnxruntime
import random
from torch import nn
import torch
import torch.nn.functional as F

seed = 0
torch.manual_seed(seed)
random.seed(seed)
np.random.seed(seed)
onnxruntime.set_seed(seed)
if torch.cuda.is_available():
    torch.cuda.manual_seed_all(seed)

class Model(nn.Module):
    def forward(self, x):
        p = F.pad(x, (1, 1), mode="constant").squeeze(1)
        return dict(o=p)

model = Model()
model.to(device="cuda")
x = torch.rand(1, 10).to(device="cuda")
print(x.shape)
print(model(x))

torch.onnx.export(
    model,
    (x,),
    "fp32_squeeze_repro.onnx",
    verbose=True,
    opset_version=16,
    export_params=False,
    do_constant_folding=True,
    input_names=["x"],
    output_names=["o"],
    dynamic_axes={"x": [0, 1], "o": [0, 1]},  # COMMENTING OUT RAISES AN EXCEPTION
)

runtime = onnxruntime.InferenceSession(
    "fp32_squeeze_repro.onnx",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)

print("pt output:", model(x))
print("ort output:", runtime.run(None, dict(x=x.cpu().numpy())))

With dynamic_axes={"x": [0, 1], "o": [0, 1]}:

python /home/thiagofc/dev/pytorch_repros/missing_complex.py
torch.Size([1, 10])
{'o': tensor([[0.0000, 0.4963, 0.7682, 0.0885, 0.1320, 0.3074, 0.6341, 0.4901, 0.8964,
         0.4556, 0.6323, 0.0000]], device='cuda:0')}
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:2039: UserWarning: No names were found for specified dynamic axes of provided input.Automatically generated names will be applied to each dynamic axes of input x
  warnings.warn(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:2039: UserWarning: No names were found for specified dynamic axes of provided input.Automatically generated names will be applied to each dynamic axes of input o
  warnings.warn(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/_internal/jit_utils.py:258: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:687: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1891.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:687: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:1178: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1891.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:1178: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_graph_shape_type_inference(
Exported graph: graph(%x : Float(*, *, strides=[10, 1], requires_grad=0, device=cuda:0)):
  %onnx::Pad_2 : NoneType = prim::Constant(), scope: __main__.Model::
  %/Constant_output_0 : Long(1, strides=[1], requires_grad=0, device=cpu) = onnx::Constant[value={2}, onnx_name="/Constant"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_1_output_0 : Long(2, strides=[1], requires_grad=0, device=cpu) = onnx::Constant[value= 1  1 [ CPULongType{2} ], onnx_name="/Constant_1"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/ConstantOfShape_output_0 : Long(2, strides=[1], device=cpu) = onnx::ConstantOfShape[value={0}, onnx_name="/ConstantOfShape"](%/Constant_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Concat_output_0 : Long(4, strides=[1], device=cpu) = onnx::Concat[axis=0, onnx_name="/Concat"](%/Constant_1_output_0, %/ConstantOfShape_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_2_output_0 : Long(2, strides=[1], device=cpu) = onnx::Constant[value=-1  2 [ CPULongType{2} ], onnx_name="/Constant_2"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Reshape_output_0 : Long(2, 2, strides=[2, 1], device=cpu) = onnx::Reshape[allowzero=0, onnx_name="/Reshape"](%/Concat_output_0, %/Constant_2_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_3_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={0}, onnx_name="/Constant_3"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_4_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-1}, onnx_name="/Constant_4"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_5_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-9223372036854775807}, onnx_name="/Constant_5"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_6_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-1}, onnx_name="/Constant_6"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Slice_output_0 : Long(2, 2, strides=[2, 1], device=cpu) = onnx::Slice[onnx_name="/Slice"](%/Reshape_output_0, %/Constant_4_output_0, %/Constant_5_output_0, %/Constant_3_output_0, %/Constant_6_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Transpose_output_0 : Long(2, 2, strides=[2, 1], device=cpu) = onnx::Transpose[perm=[1, 0], onnx_name="/Transpose"](%/Slice_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_7_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-1}, onnx_name="/Constant_7"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Reshape_1_output_0 : Long(4, strides=[1], device=cpu) = onnx::Reshape[allowzero=0, onnx_name="/Reshape_1"](%/Transpose_output_0, %/Constant_7_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Cast_output_0 : Long(4, strides=[1], device=cpu) = onnx::Cast[to=7, onnx_name="/Cast"](%/Reshape_1_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Pad_output_0 : Float(*, *, strides=[12, 1], requires_grad=0, device=cuda:0) = onnx::Pad[mode="constant", onnx_name="/Pad"](%x, %/Cast_output_0, %onnx::Pad_2), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_8_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={1}, onnx_name="/Constant_8"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Shape_output_0 : Long(2, strides=[1], device=cpu) = onnx::Shape[onnx_name="/Shape"](%/Pad_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Gather_output_0 : Long(1, strides=[1], device=cpu) = onnx::Gather[axis=0, onnx_name="/Gather"](%/Shape_output_0, %/Constant_8_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_9_output_0 : Long(1, strides=[1], requires_grad=0, device=cpu) = onnx::Constant[value={1}, onnx_name="/Constant_9"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Equal_output_0 : Bool(1, strides=[1], device=cpu) = onnx::Equal[onnx_name="/Equal"](%/Gather_output_0, %/Constant_9_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %o : Float(*, device=cpu) = onnx::If[onnx_name="/If"](%/Equal_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
    block0():
      %/Constant_10_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={1}, onnx_name="/Constant_10"](), scope: __main__.Model::
      %/Squeeze_output_0 : Float(*, device=cpu) = onnx::Squeeze[onnx_name="/Squeeze"](%/Pad_output_0, %/Constant_10_output_0), scope: __main__.Model::
      -> (%/Squeeze_output_0)
    block1():
      %/Identity_output_0 : Float(*, *, device=cpu) = onnx::Identity[onnx_name="/Identity"](%/Pad_output_0), scope: __main__.Model::
      -> (%/Identity_output_0)
  return (%o)

pt output: {'o': tensor([[0.0000, 0.4963, 0.7682, 0.0885, 0.1320, 0.3074, 0.6341, 0.4901, 0.8964,
         0.4556, 0.6323, 0.0000]], device='cuda:0')}
2022-11-14 17:20:56.739266451 [W:onnxruntime:, execution_frame.cc:828 VerifyOutputSizes] Expected shape from model of {-1} does not match actual shape of {1,12} for output o
ort output: [array([[0.        , 0.4962566 , 0.7682218 , 0.08847743, 0.13203049,
        0.30742282, 0.6340787 , 0.4900934 , 0.89644474, 0.45562798,
        0.6323063 , 0.        ]], dtype=float32)]

Without dynamic_axes={"x": [0, 1], "o": [0, 1]}:

python /home/thiagofc/dev/pytorch_repros/missing_complex.py
torch.Size([1, 10])
{'o': tensor([[0.0000, 0.4963, 0.7682, 0.0885, 0.1320, 0.3074, 0.6341, 0.4901, 0.8964,
         0.4556, 0.6323, 0.0000]], device='cuda:0')}
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/_internal/jit_utils.py:258: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:687: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1891.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:687: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:1178: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1891.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/thiagofc/dev/github/pytorch-dev1/torch/onnx/utils.py:1178: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at /home/thiagofc/dev/github/pytorch-dev1/torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_graph_shape_type_inference(
Exported graph: graph(%x : Float(1, 10, strides=[10, 1], requires_grad=0, device=cuda:0)):
  %onnx::Pad_2 : NoneType = prim::Constant(), scope: __main__.Model::
  %/Constant_output_0 : Long(1, strides=[1], requires_grad=0, device=cpu) = onnx::Constant[value={2}, onnx_name="/Constant"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_1_output_0 : Long(2, strides=[1], requires_grad=0, device=cpu) = onnx::Constant[value= 1  1 [ CPULongType{2} ], onnx_name="/Constant_1"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/ConstantOfShape_output_0 : Long(2, strides=[1], device=cpu) = onnx::ConstantOfShape[value={0}, onnx_name="/ConstantOfShape"](%/Constant_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Concat_output_0 : Long(4, strides=[1], device=cpu) = onnx::Concat[axis=0, onnx_name="/Concat"](%/Constant_1_output_0, %/ConstantOfShape_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_2_output_0 : Long(2, strides=[1], device=cpu) = onnx::Constant[value=-1  2 [ CPULongType{2} ], onnx_name="/Constant_2"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Reshape_output_0 : Long(2, 2, strides=[2, 1], device=cpu) = onnx::Reshape[allowzero=0, onnx_name="/Reshape"](%/Concat_output_0, %/Constant_2_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_3_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={0}, onnx_name="/Constant_3"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_4_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-1}, onnx_name="/Constant_4"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_5_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-9223372036854775807}, onnx_name="/Constant_5"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_6_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-1}, onnx_name="/Constant_6"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Slice_output_0 : Long(2, 2, strides=[2, 1], device=cpu) = onnx::Slice[onnx_name="/Slice"](%/Reshape_output_0, %/Constant_4_output_0, %/Constant_5_output_0, %/Constant_3_output_0, %/Constant_6_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Transpose_output_0 : Long(2, 2, strides=[2, 1], device=cpu) = onnx::Transpose[perm=[1, 0], onnx_name="/Transpose"](%/Slice_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_7_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={-1}, onnx_name="/Constant_7"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Reshape_1_output_0 : Long(4, strides=[1], device=cpu) = onnx::Reshape[allowzero=0, onnx_name="/Reshape_1"](%/Transpose_output_0, %/Constant_7_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Cast_output_0 : Long(4, strides=[1], device=cpu) = onnx::Cast[to=7, onnx_name="/Cast"](%/Reshape_1_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Pad_output_0 : Float(*, *, strides=[12, 1], requires_grad=0, device=cuda:0) = onnx::Pad[mode="constant", onnx_name="/Pad"](%x, %/Cast_output_0, %onnx::Pad_2), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_8_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={1}, onnx_name="/Constant_8"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Shape_output_0 : Long(2, strides=[1], device=cpu) = onnx::Shape[onnx_name="/Shape"](%/Pad_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Gather_output_0 : Long(1, strides=[1], device=cpu) = onnx::Gather[axis=0, onnx_name="/Gather"](%/Shape_output_0, %/Constant_8_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Constant_9_output_0 : Long(1, strides=[1], requires_grad=0, device=cpu) = onnx::Constant[value={1}, onnx_name="/Constant_9"](), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %/Equal_output_0 : Bool(1, strides=[1], device=cpu) = onnx::Equal[onnx_name="/Equal"](%/Gather_output_0, %/Constant_9_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
  %o : Float(*, device=cpu) = onnx::If[onnx_name="/If"](%/Equal_output_0), scope: __main__.Model:: # /home/thiagofc/dev/pytorch_repros/missing_complex.py:20:0
    block0():
      %/Constant_10_output_0 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={1}, onnx_name="/Constant_10"](), scope: __main__.Model::
      %/Squeeze_output_0 : Float(*, device=cpu) = onnx::Squeeze[onnx_name="/Squeeze"](%/Pad_output_0, %/Constant_10_output_0), scope: __main__.Model::
      -> (%/Squeeze_output_0)
    block1():
      %/Identity_output_0 : Float(*, *, device=cpu) = onnx::Identity[onnx_name="/Identity"](%/Pad_output_0), scope: __main__.Model::
      -> (%/Identity_output_0)
  return (%o)

Traceback (most recent call last):
  File "/home/thiagofc/dev/pytorch_repros/missing_complex.py", line 43, in <module>
    runtime = onnxruntime.InferenceSession(
  File "/home/thiagofc/dev/github/onnxruntime-dev1/build/Linux/RelWithDebInfo/onnxruntime/capi/onnxruntime_inference_collection.py", line 360, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/thiagofc/dev/github/onnxruntime-dev1/build/Linux/RelWithDebInfo/onnxruntime/capi/onnxruntime_inference_collection.py", line 408, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (/If) Op (If) [TypeInferenceError] Graph attribute inferencing failed: Node (/Squeeze) Op (Squeeze) [ShapeInferenceError] Dimension of input 1 must be 1 instead of 12
vadimkantorov commented 6 months ago

@thiagocrepaldi Is the last item still pending?

Are plenty of Gather/Constant/Equal nodes from the F.pad export implementation? This seems quite a long nasty graph just for a single padding op :( Is it to compensate semantic difference from Pad op? I wonder if it's possible to get a nicer graph

thiagocrepaldi commented 6 months ago

I am not sure. IIRC the exporter relies on onnx shape inference to validate and propagate the shape for the if. the exporter itself cannot make any guarantees there

However, torch.onnx.export is in maintenance mode and we don't plan to add new operators/features or fix complex issues.

Please try the new ONNX exporter and reopen this issue with a full repro if it also doesn't work for you: quick torch.onnx.dynamo_export API tutorial

vadimkantorov commented 6 months ago

Regarding the new exporter, two big questions are standing... 1) support for RNN export <- this was a regression 2) support for complex tensors and op lowering, important for torch fft/complex ops...