search

tenstorrent / tt-forge-fe

The TT-Forge FE is a graph compiler designed to optimize and transform computational graphs for deep learning models, enhancing their performance and efficiency.
https://docs.tenstorrent.com/tt-forge-fe/
Apache License 2.0
6 stars 1 forks source link

[Bug] Unsupported PCC calculation for shape (1,1) #290

Open kmilanovicTT opened 1 week ago

kmilanovicTT commented 1 week ago

Description

When testing binary ops with PyTorch models in inference mode tests are only failing when using shape: (1,1) for input tensor, regardless of used model(operand source). This is for all implemented binary ops (add, div, mul, true_divide, ge, sub).

Error message is: E AssertionError: PCC for single values doesn't work

Reproduce:

Run command pytest -svv forge/test/operators_pytorch/eltwise_binary/test_pytorch_binary.py::test_pytorch_eltwise_binary_ops_per_test_plan --runxfail

vbrkicTT commented 2 days ago

@nvukobratTT @dgolubovicTT compare_with_golden_pcc doesn't work for single values

dgolubovicTT commented 2 days ago

Hi, as discussed on the meeting, PCC is Pearson Correlation Coefficient. It is not deffined for single values. There is the function that is called in case of single values. Use compare_with_golden and it will call compare_with_golden_pcc or torch.allclose depending on the size of the tensor. This should be used throughout the forge testing since it enables single values. I will make the same change in forge/test/mlir/test_ops.py. Thanks for the catch

vbrkicTT commented 1 day ago

Tnx @dgolubovicTT