tenstorrent / tt-mlir

Tenstorrent MLIR compiler
https://tenstorrent.github.io/tt-mlir/
Apache License 2.0
66 stars 8 forks source link

Consider introducing scalars into the TTIR\TTNN #767

Open sdjordjevicTT opened 1 month ago

sdjordjevicTT commented 1 month ago

Some ops can produce scalar values, like Matmul vector-vector product: https://github.com/tenstorrent/tt-mlir/pull/745#discussion_r1764954874

Today, we are converting scalar values into 1D tensors with a size of 1. Perhaps we can consider supporting 0 rank tensors(tensor<f32>, not f32). This is a special case that needs to be handled when constructing and passing into the runtime, but it may not be a significant issue.

nsmithtt commented 1 month ago

So I think we should make the distinction here, that we can potentially support 0 rank tensors:

sdjordjevicTT commented 1 month ago

Fully agree, updated initial issue description.