tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
396 stars 48 forks source link

Maxpool fails in Model_net #11160

Open HariniMohan0102 opened 1 month ago

HariniMohan0102 commented 1 month ago

Describe the bug Here is the unit tests of failing maxpool op of each resolution:

When Maxpooling=True, Encoder res: 4094x510

When Maxpooling=True, Encoder res: 2047x255

To Reproduce Steps to reproduce the behavior:

  1. Checkout to the branch harini/model_net_failing_maxpools
  2. Run the respective commands of Ops in each input resolution to reproduce the issues.

Expected behavior To run the op for the specific input configurations without error.

Please complete the following environment information:

sankarmanoj-tt commented 4 weeks ago

Maxpool supports only bfloat16. So any bfloat8 inputs must be converted to bfloat16. My guess is that when this conversion happens, there isn't enough memory to fit both input and output tensors.

mywoodstock commented 4 weeks ago

Yeah, basically you can provide bfp8_b input to the maxpool op. Internally the halo op converts to RM, so BFP16. If input is already BFP16, it should need less memory.