nod-ai / sharktank

SHARK Inference Modeling and Serving
Apache License 2.0
7 stars 9 forks source link

Sharded unreduced tensor #57

Closed sogartar closed 4 weeks ago

sogartar commented 4 weeks ago

I am thinking of adding another sharded tensor type that represents a sharded tensor, that need to be reduced (its shards summed) to get the actual tensor. I see that the point of this is to reorder the summation to be done after a subsequent operation as shown here. This information should be carried in the type so that downstream ops can dispatch on the type and decide whether to sum before or after.

sogartar commented 4 weeks ago

Here is a PR that implements the type #59.