tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
471 stars 74 forks source link

tt_lib.tensor.rpow fails with low PCC [Grayskull] [Doc] #3309

Closed npetrovic-tenstorrent closed 1 week ago

npetrovic-tenstorrent commented 1 year ago

Describe the bug

ttl.tensor.rpow operation breaks with low PCC error in some test cases

To Reproduce

Steps to reproduce the behavior:

Checkout main branch Run unit test test_eltwise_rpow.py using this command: pytest tests/tt_eager/python_api_testing/non_working_unit_tests/grayskull/test_eltwise_rpow.py Expected behavior

There are test cases presented in the unit test test_eltwise_rpow.py and most of them are expected to fail with low PCC error.

For example, one of the tests is expected to fail with this result: Max ATOL Delta: nan, Max RTOL Delta: nan, PCC: 0.9267387350542515, PCC check failed

Getting Additional info for the operation under test and its behavior

To get additional information and results for different combinations of input shapes, types, layouts and memory configs for which this operation was tested you can also run locally sweeps for ttl.tensor.ne and check the results. To do this you should:

Follow the Getting Started page to setup the repo, environment variables and python-env Activate source build/python_env/bin/activate Run sweeps by using python tests/tt_eager/python_api_testing/sweep_tests/run_pytorch_test.py -i tests/tt_eager/python_api_testing/sweep_tests/test_configs/ci_sweep_tests/broken_grayskull/pytorch_eltwise_rpow_test.yaml -o ./result-sweeps After the run is completed all test sweeps results should be available inside specified output directory (in this case ./result-sweeps). There you will find eltwise_rpow_sweep.csv which holds all executed sweeps, among which you can also find the ones that failed and were recreated by the unit test, which you can get by searching unique data_seed field.

bharane-ab commented 10 months ago

As Rpow is computed using multiple functions like logarithm, broadcasting, and exponentiation so it is supported for various range based on the factor(base) value.

Image

npetrovic-tenstorrent commented 7 months ago

It still seems to exhibit low PCC in range [-100, 100]

bharane-ab commented 7 months ago

Hi @jliangTT @eyonland @tt-aho @npetrovic-tenstorrent

As mentioned here, Rpow is computed using the log, mul and exp operations. To support a range from -100 to 85, the base value should be 2. So for different base value the input range will be different.

Kindly give your suggestion on this.

npetrovic-tenstorrent commented 5 months ago

Hi @jliangTT @eyonland @tt-aho @npetrovic-tenstorrent

As mentioned here, Rpow is computed using the log, mul and exp operations. To support a range from -100 to 85, the base value should be 2. So for different base value the input range will be different.

Kindly give your suggestion on this.

These limitations should be clearly stated in documentation.

npetrovic-tenstorrent commented 5 months ago

As Rpow is computed using multiple functions like logarithm, broadcasting, and exponentiation so it is supported for various range based on the factor(base) value.

Image

Some form of these info should be added to the documentation, so users would be aware of its usage limitations.

KalaivaniMCW commented 3 months ago

Will revisit this task and update it after migration task

ruthreshx commented 2 months ago

Hi @npetrovic-tenstorrent , Sure we we will add the documentation along with the usage limitations.

mcw-anasuya commented 3 weeks ago

Rpow supports an input range upto 25.

Screenshot 2024-10-25 at 16 43 22
eyonland commented 2 weeks ago

Please confirm this issue on WH and then update the documentation as mentioned above.

mcw-anasuya commented 2 weeks ago

@eyonland Checked in WH, the input range supports upto 28. The document has been updated and merged to main.

Screenshot 2024-10-30 at 20 45 51
mcw-anasuya commented 1 week ago

The PCC is 0.99 for input range up to 28, which is good for WH. So I'm closing this issue.