search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
https://docs.tenstorrent.com/ttnn/latest/index.html
Apache License 2.0
488 stars 81 forks source link

Expose Conv2D weight/bias preparation #12793

Open LPanosTT opened 2 months ago

LPanosTT commented 2 months ago

Is your feature request related to a problem? Please describe. So long as the weights/biases are formatted and placed on device by the Conv2D op itself, there is no simple way to be able to push weights onto device in MLIR before execution - leading to a slow execution.

Describe the solution you'd like For MLIR we'd like to be able to prepare Conv2D weights as their own op, and then pass the on-device tensors to the Conv2D op.

Describe alternatives you've considered I've considered just executing the Conv2D beforehand and then somehow holding on to a reference to the weights to use later. But This would likely require a significant amount of special-case logic that isn't necessary. It would be much cleaner if all weights for a given compiled model could be ready to push to device and use without any transformations before their respective operation.

Additional context I'm working on bringing up tensorflow Resnet50 through forge-fe --> MLIR --> ttnn runtime, so we'll have an actual model for which this would be useful for soon.

CC: @nsmithtt @nvukobratTT @mywoodstock

LPanosTT commented 2 months ago

Removed feature request label since this is an internal feature request.

LPanosTT commented 1 month ago

Being resolved in: https://github.com/tenstorrent/tt-metal/pull/14049

LPanosTT commented 1 month ago

Done

LPanosTT commented 1 month ago

Some perf tests are broken, reverting