Open rtawfik01 opened 1 month ago
Measured perf for pack untilize block test with following args: block C dim: 4 tiles block R dim: 1 tile Num cores: 1
To reproduce:
git checkout rtawfik/pack_untilize_perf
ENABLE_TRACY=1 scripts/build_scripts/build_with_profiler_opt.sh
ninja tests -C build
ENABLE_TRACY=1 TT_METAL_DEVICE_PROFILER=1 TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests --gtest_filter=“*ComputePackUntilize*”
And the Tracy GUI results can be seen here:
Cycles can be calculated using the machines AICLK and The GPU execution time
The branch has the blocking calls for circular buffers and math-pack semaphores commented out. The results for the number of cycles the PACK takes is:
WHB0: ~145 cycles (average 10 runs) BH: ~155 cycles (average 10 runs)
The results consistently show BH and WHB0 have around the same cycles count difference for block of tiles less than 6. For c_dim > 6, unfortunately Wormhole B0 has a smaller depth instruction buffer (16 insns), in comparison to Blackhole (32 insns), and that affects the results since at around 6 tiles is when math issues 16 insns and needs to stall for wormhole b0. Better results can be obtained from waveforms.
@ttmtrajkovic fyi
Pack untilize kernel needed to be re-written for Blackhole, due to Blackhole packer limitations:
Here is the current algorithm:![image](https://github.com/user-attachments/assets/dd4ac962-3371-4b83-84a9-78237f9a6362)
For wormhole b0, 8x16 rows can be output per tile (before incrementing counters), while for blackhole with the above implementation, only 2x16 can. Since pack_untilize feature needs to be fused with other operations, we cannot change the unpacker to use a different unpacking scheme (Fastest untilize scheme would be T0F0, T0F1, T1F0, T1F1, T0F2, T0F3, T1F2, T1,F3)). However, we should use that implementation if the use case is doing a pack_untilize for a block of tiles (i.e not fused).
One major optimization, is to enable the use of contexts.
THCON_SEC0_REG1_L1_Dest_addr_ADDR32 = CNTX0 = tile_offset (Configured for top faces) THCON_SEC0_REG8_L1_Dest_addr_ADDR32 = CNTX2 = tile_offset + block_ct_dim TILE_C_DIM datum_size ((Configured for bottom faces)
Then algorithm can do 4x16 rows, will change to something like this:
The problem here is if contexts are used, initial hardware configuration for data formats, tile sizes, etc, all need to also be programmed.
@ttmtrajkovic @rdjogoTT fyi