search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
400 stars 50 forks source link

Replace back to back sharded2interleaved and interleaved2sharded in SD with reshard #6182

Open AleksKnezevic opened 6 months ago

AleksKnezevic commented 6 months ago

Please replace all the back to back conversions to interleaved and back to sharded with reshard.

In order to do this safely, we should first collect all the conversion shard specs, and create unit tests for reshard, and once those unit tests pass, make the modification in the model.

AleksKnezevic commented 5 months ago

@mtatsumiTT, can you please update with progress?

mtatsumiTT commented 5 months ago

I collected input/output configs for reshard op in SD model and tested, and replacing the sequence with reshard op is currently blocked either 1KB issue (#6206) or shard-spec error as following:

E       RuntimeError: TT_FATAL @ tt_eager/tt_dnn/op_library/sharded/sharded_op.cpp:91: same_row_size
E       info:
E       row major must have shard_spec[1] be the same on both input and output

However, the shard-spec of conv2d ops would likely change after @tt-nshanker 's conv2d 2.0 so we have not asked for specific update for reshard op yet.

nsmithtt commented 4 months ago

@AleksKnezevic, this is complete right?

AleksKnezevic commented 4 months ago

No, we still have some unsupported reshards that @tarafdarTT is working on.