TTNN throws an exception if second operand of binary Ops require broadcast

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

Apache License 2.0

453 stars 66 forks source link

Open uazizTT opened 2 weeks ago

uazizTT commented 2 weeks ago

TTNN throws the following exception when trying to broadcast second operand of a binary op:

RuntimeError: TT_THROW @ tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/eltwise/binary/device/binary_device_operation.cpp:46: tt::exception

TT_THROW("ttnn::operations::binary::BinaryDeviceOperation: unsupported broadcast");

Apply canonicalization on the operands or allow broadcasting for all operands.

eyonland commented 2 weeks ago

Please attach a pytest to reproduce this exact case.

Also, note that the workaround we currently have for broadcasting is to use the ttnn.bcast. For example https://github.com/tenstorrent/tt-metal/blob/main/models/demos/t3000/falcon40b/tt/model_utils.py#L504