Among 7 failed convs, 4 Convs fails with OOM error, 2 Convs failed with the following error: RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
The other conv fails with the error RuntimeError: shape '[32, 2500, 3]' is invalid for input of size 2560000
Input Resolution: (32,3,1024)
7 ttnn Conv1d ops failed.
Among 7 failed convs, 4 Convs fails with OOM error, 2 Convs failed with the following error: RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
The other conv fails with the error RuntimeError: shape '[32, 1024, 3]' is invalid for input of size 1048576
Input Resolution: (32,3,512)
6 ttnn Conv1d ops failed.
Among 6 failed convs, 2 Convs failed with the following error: RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
1 conv fails with the error RuntimeError: shape '[32, 512, 3]' is invalid for input of size 524288
2 convs fails when output dtype is bfloat16 with RuntimeError: TT_THROW @ ../tt_metal/impl/program/program.cpp:519 tt::exception
1 conv fails with OOM error.
Input Resolution: (32,3,256)
4 ttnn Conv1d ops failed.
Among 4 failed convs, 2 Convs failed with the following error: RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
1 conv fails with the error RuntimeError: shape '[32, 256, 3]' is invalid for input of size 262144
1 convs fails with RuntimeError: TT_THROW @ ../tt_metal/impl/program/program.cpp:519 tt::exception
Input Resolution: (32,3,128)
4 ttnn Conv1d ops failed.
Among 4 failed convs, 2 Convs failed with the following error: RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
1 conv fails with the error RuntimeError: shape '[32, 128, 3]' is invalid for input of size 131072
1 convs fails when output dtype is bfloat16 with RuntimeError: TT_THROW @ ../tt_metal/impl/program/program.cpp:519 tt::exception
Run the command to unit test conv1d of resolution (32,3,2500): pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d
Run the command to unit test conv1d of resolution (32,3,1024): pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_1024
Run the command to unit test conv1d of resolution (32,3,512): pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_512
Run the command to unit test conv1d of resolution (32,3,256): pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_256
Run the command to unit test conv1d of resolution (32,3,128): pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_128
Expected behavior
Expected to pass the failing unit test configurations in conv1d.
Please complete the following environment information:
Describe the bug
Input Resolution: (32,3,2500)
RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
RuntimeError: shape '[32, 2500, 3]' is invalid for input of size 2560000
Input Resolution: (32,3,1024)
RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
RuntimeError: shape '[32, 1024, 3]' is invalid for input of size 1048576
Input Resolution: (32,3,512)
RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
RuntimeError: shape '[32, 512, 3]' is invalid for input of size 524288
RuntimeError: TT_THROW @ ../tt_metal/impl/program/program.cpp:519 tt::exception
Input Resolution: (32,3,256)
RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
RuntimeError: shape '[32, 256, 3]' is invalid for input of size 262144
RuntimeError: TT_THROW @ ../tt_metal/impl/program/program.cpp:519 tt::exception
Input Resolution: (32,3,128)
RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/operations/matmul/device/matmul_op.cpp:959: a_shape[-1] == b_shape[-2]
RuntimeError: shape '[32, 128, 3]' is invalid for input of size 131072
RuntimeError: TT_THROW @ ../tt_metal/impl/program/program.cpp:519 tt::exception
To Reproduce Steps to reproduce the behavior:
pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d
pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_1024
pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_512
pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_256
pytest tests/ttnn/unit_tests/operations/test_conv1d.py::test_pointnet_conv1d_low_res_128
Expected behavior Expected to pass the failing unit test configurations in conv1d.
Please complete the following environment information: