search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
421 stars 54 forks source link

UNet with different input resolutions bringup #8031

Open dvartaniansTT opened 5 months ago

dvartaniansTT commented 5 months ago

Describe the solution you'd like Bringup the shallow UNet variant with the following input resolutions:

  1. 256x512
  2. 512x512
  3. 1024x1024

Additional context this should be developed on WH (N150)

keerthana-r-mcw commented 4 months ago

Hi, I pushed the changes for 256x512 resolution support. I am attaching the perf sheet for the same. ops_perf_results_shallow_unet_2024_05_09_10_21_15.csv

keerthana-r-mcw commented 4 months ago

Hi, I tried running shallow unet 256x512 with group=2 and batch=2 and It is failing the following error, RuntimeError: TT_THROW @ tt_metal/impl/allocator/allocator.cpp:118: tt::exception E info: E Out of Memory: Not enough space to allocate {} B {} buffer across {} banks, where each bank needs to store {} B E 33554432 E L1 E 64 E 524288

occuring at models/experimental/functional_unet/tt/ttnn_shallow_unet.py:256: in __call__ output_tensor = unet_concat([output_tensor, save_c1_2_out], dim=-1, perf_mode=perf_mode)

keerthana-r-mcw commented 4 months ago

I am attaching the profiler for shallow unet 256x512 with group=2 and batch=1. ops_perf_results_unet_2024_05_14_14_55_02.csv

dvartaniansTT commented 4 months ago

@keerthana-r-mcw Could you please attach the command you ran to generate this sheet here?