justinchuby
commented
3 months ago PyTorch ONNX Conversion Error Report
✅ Obtain model graph with `torch.export.export`
❌ Translate the graph into ONNX
⚪ Run `onnx.checker` on the ONNX model
⚪ Execute the model with ONNX Runtime
⚪ Validate model output accuracyError message:
Traceback (most recent call last):
File "/Users/justinc/Documents/GitHub/torch-onnx/src/torch_onnx/_core.py", line 490, in _add_nodes
_handle_call_function_node_with_lowering(
File "/Users/justinc/Documents/GitHub/torch-onnx/src/torch_onnx/_core.py", line 372, in _handle_call_function_node_with_lowering
raise RuntimeError(
RuntimeError: No ONNX function found for <OpOverload(op='aten.addmm', overload='default')>. Failure message: No decompositions registered for the real-valued input
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/justinc/Documents/GitHub/torch-onnx/src/torch_onnx/_patch.py", line 174, in _torch_onnx_export_adaptor
ir_model = torch_onnx.exported_program_to_ir(program)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/justinc/Documents/GitHub/torch-onnx/src/torch_onnx/_core.py", line 626, in exported_program_to_ir
values = _add_nodes(exported_program, model, lower=lower, registry=registry)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/justinc/Documents/GitHub/torch-onnx/src/torch_onnx/_core.py", line 501, in _add_nodes
raise RuntimeError(
RuntimeError: Error when translating node %addmm : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%arg6_1, %view_1, %t), kwargs = {}). See the stack trace for more information.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/justinc/Documents/GitHub/torch-onnx/src/torch_onnx/_patch.py", line 230, in _torch_onnx_export_adapter_with_error_report
ir_model, program = _torch_onnx_export_adaptor(*args, **kwargs, check=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/justinc/Documents/GitHub/torch-onnx/src/torch_onnx/_patch.py", line 197, in _torch_onnx_export_adaptor
raise OnnxConversionError(
torch_onnx._patch.OnnxConversionError: Failed to convert the exported program to an ONNX model. [96mThis is step 2/2[0m of exporting the model to ONNX. Please create an issue in the PyTorch GitHub repository against the [96m*onnx*[0m component and attach the full error stack as well as reproduction scripts. You can run `torch_onnx.analyze()` to produce an error report after obtaining an ExportedProgram with `torch.export.export()`.Exported program:
Analysis
PyTorch ONNX Conversion Analysis
Model Information
The model has 127398144 parameters and 0 buffers (non-trainable parameters). Number of parameters per dtype:
defaultdict(<class 'int'>, {torch.float32: 127398144})Number of buffers per dtype:
defaultdict(<class 'int'>, {})Inputs:
arg199_1:TensorMetadata(shape=torch.Size([1, 12]), dtype=torch.int64, requires_grad=False, stride=(12, 1), memory_format=torch.contiguous_format, is_quantized=False, qparams={})arg200_1:TensorMetadata(shape=torch.Size([1, 12]), dtype=torch.int64, requires_grad=False, stride=(12, 1), memory_format=torch.contiguous_format, is_quantized=False, qparams={})
Outputs:
slice_2778:TensorMetadata(shape=torch.Size([1, 12, 768]), dtype=torch.float32, requires_grad=False, stride=(393216, 768, 1), memory_format=torch.contiguous_format, is_quantized=False, qparams={})tanh:TensorMetadata(shape=torch.Size([1, 768]), dtype=torch.float32, requires_grad=False, stride=(768, 1), memory_format=torch.contiguous_format, is_quantized=False, qparams={})
The FX graph has 5064 nodes in total. Number of FX nodes per op:
placeholder: 201call_function: 4862output: 1
Of the call_function nodes, the counts of operators used are:
aten.slice.Tensor: 1627aten.view.default: 744aten.slice_scatter.default: 528aten.transpose.int: 420aten.unsqueeze.default: 170aten.select.int: 147aten.permute.default: 144aten.copy.default: 144aten.t.default: 73aten.addmm.default: 73aten._to_copy.default: 64aten.clone.default: 61aten.as_strided.default: 60aten.constant_pad_nd.default: 51aten.select_scatter.default: 48aten.flip.default: 48aten.expand.default: 48aten.full_like.default: 48aten.where.self: 48aten.add.Tensor: 39aten.new_ones.default: 36aten.native_layer_norm.default: 25<built-in function getitem>: 25aten.bmm.default: 24aten.new_zeros.default: 24aten.tril.default: 24aten.masked_fill.Scalar: 24aten._unsafe_view.default: 24aten.mul.Tensor: 14aten.ne.Scalar: 13aten.div.Tensor: 12aten._softmax.default: 12aten.gelu.default: 12aten.embedding.default: 3aten.zeros.default: 1aten.rsub.Scalar: 1aten.cumsum.default: 1aten.lt.Scalar: 1aten.tanh.default: 1
ONNX Conversion Information
The model contains operators the dispatcher could not find registered ONNX decompositions for. This may be due to missing implementations, decompositions not registered correctly, or a bug in the dispatcher.
Errors grouped by operator:
aten.addmm.default: No decompositions registered for the real-valued input. Example node:%addmm : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%arg6_1, %view_1, %t), kwargs = {}). All nodes:[addmm, addmm_1, addmm_2, addmm_3, addmm_4, addmm_5, addmm_6, addmm_7, addmm_8, addmm_9, addmm_10, addmm_11, addmm_12, addmm_13, addmm_14, addmm_15, addmm_16, addmm_17, addmm_18, addmm_19, addmm_20, addmm_21, addmm_22, addmm_23, addmm_24, addmm_25, addmm_26, addmm_27, addmm_28, addmm_29, addmm_30, addmm_31, addmm_32, addmm_33, addmm_34, addmm_35, addmm_36, addmm_37, addmm_38, addmm_39, addmm_40, addmm_41, addmm_42, addmm_43, addmm_44, addmm_45, addmm_46, addmm_47, addmm_48, addmm_49, addmm_50, addmm_51, addmm_52, addmm_53, addmm_54, addmm_55, addmm_56, addmm_57, addmm_58, addmm_59, addmm_60, addmm_61, addmm_62, addmm_63, addmm_64, addmm_65, addmm_66, addmm_67, addmm_68, addmm_69, addmm_70, addmm_71, addmm_72]
Profile result
_ ._ __/__ _ _ _ _ _/_ Recorded: 10:45:15 Samples: 21751
/_//_/// /_\ / //_// / //_'/ // Duration: 23.412 CPU time: 27.537
/ _/ v4.6.2
Program: /Users/justinc/Documents/GitHub/torch-onnx/tests/longformer_export_api.py
23.411 _torch_onnx_export_adapter_with_error_report torch_onnx/_patch.py:210
└─ 23.411 _torch_onnx_export_adaptor torch_onnx/_patch.py:139
├─ 14.097 export torch/export/__init__.py:73
│ [345 frames hidden] torch, <string>, contextlib, importli...
└─ 9.314 exported_program_to_ir torch_onnx/_core.py:574
├─ 8.100 wrapper torch/export/exported_program.py:80
│ [102 frames hidden] torch, <string>
├─ 0.853 insert_type_promotion_nodes torch_onnx/_fx_passes.py:6
│ └─ 0.795 wrapper torch/onnx/_internal/diagnostics/infra/decorator.py:71
│ [14 frames hidden] torch
└─ 0.322 OnnxRegistry.from_torchlib torch_onnx/_registration.py:114
E.g
PyTorch ONNX Conversion Error Report
Error message:
Exported program:
Analysis
PyTorch ONNX Conversion Analysis
Model Information
The model has 0 parameters and 0 buffers (non-trainable parameters). Number of parameters per dtype: defaultdict(<class 'int'>, {}) Number of buffers per dtype: defaultdict(<class 'int'>, {})
Inputs:
arg0_1:TensorMetadata(shape=torch.Size([1, 1, 2, 3, 2]), dtype=torch.float32, requires_grad=False, stride=(12, 12, 6, 2, 1), memory_format=torch.contiguous_format, is_quantized=False, qparams={})arg1_1:TensorMetadata(shape=torch.Size([1, 3, 2, 4, 3]), dtype=torch.float32, requires_grad=False, stride=(72, 24, 12, 3, 1), memory_format=torch.contiguous_format, is_quantized=False, qparams={})Outputs:
grid_sampler_3d:TensorMetadata(shape=torch.Size([1, 1, 3, 2, 4]), dtype=torch.float32, requires_grad=False, stride=(24, 24, 8, 4, 1), memory_format=torch.contiguous_format, is_quantized=False, qparams={})The FX graph has 4 nodes in total. Number of FX nodes per op:
placeholder: 2call_function: 1output: 1Of the call_function nodes, the counts of operators used are:
aten.grid_sampler_3d.default: 1ONNX Conversion Information
The model contains operators the dispatcher could not find registered ONNX decompositions for. This may be due to missing implementations, decompositions not registered correctly, or a bug in the dispatcher.
Errors grouped by operator:
aten.grid_sampler_3d.default: No decompositions registered for the real-valued input. Example node:%grid_sampler_3d : [num_users=1] = call_function[target=torch.ops.aten.grid_sampler_3d.default](args = (%arg0_1, %arg1_1, 0, 0, True), kwargs = {}). All nodes:[grid_sampler_3d]Profile result