[API Proposal]: Adding Matrix operations to `System.Numerics.Tensors.TensorPrimitives`

[shaltielshmid]

Background and motivation

Recently there's been a trend online of developing very minimal but efficient systems for LLM inference. After testing out TensorPrimitives vs the other numerical libraries out there, TensorPrimitives seems to be extremely efficient in comparison and I see an opportunity here to build a system similar to (vllm)[https://github.com/vllm-project/vllm] native to C# which can provide a solid base for LLM inferencing in C#.

API Proposal

namespace System.Numerics.Tensors {
    public static partial class TensorPrimitives {
        // Function for doing a matrix multiply
        public static float MatrixMultiply(ReadOnlySpan<float> x, ReadOnlySpan<float> y, ReadOnlySpan<float> destination);
        // Applying a softmax on a specific dimension, or perhaps the last dimension
        public static float Softmax(ReadOnlySpan<float> x, int dimension);
    }
}

API Usage

This can work for Vector / Matrix multiplication as well, by treating vectors as [1,N] matrices.

float[,] matrix1 = new float[M, K];
float[,] matrix2 = new float[K, N];
float[,] destination = new float[M, N];

TensorPrimitives.MatrixMultiply(matrix1, matrix2, destination);
// Softmax on the last "dimension", to provide probabilities for every prediction in a matrix
TensorPrimitives.Softmax(destination, -1);

These are the two main operations I can think off the top of my head, but I'll continue editing as I think of more.

@luisquintanilla

[ghost]

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors See info in area-owners.md if you want to be subscribed.

Issue Details

---

### Background and motivation Recently there's been a trend online of developing very minimal but efficient systems for LLM inference. After testing out TensorPrimitives vs the other numerical libraries out there, TensorPrimitives seems to be extremely efficient in comparison and I see an opportunity here to build a system similar to (vllm)[https://github.com/vllm-project/vllm] native to C# which can provide a solid base for LLM inferencing in C#. ### API Proposal ```csharp namespace System.Numerics.Tensors { public static partial class TensorPrimitives { // Function for doing a matrix multiply public static float MatrixMultiply(ReadOnlySpan x, ReadOnlySpan y, ReadOnlySpan destination); // Applying a softmax on a specific dimension, or perhaps the last dimension public static float Softmax(ReadOnlySpan x, int dimension); } } ``` ### API Usage This can work for Vector / Matrix multiplication as well, by treating vectors as [1,N] matrices. ```csharp float[,] matrix1 = new float[M, K]; float[,] matrix2 = new float[K, N]; float[,] destination = new float[M, N]; TensorPrimitives.MatrixMultiply(matrix1, matrix2, destination); // Softmax on the last "dimension", to provide probabilities for every prediction in a matrix TensorPrimitives.Softmax(destination, -1); ``` These are the two main operations I can think off the top of my head, but I'll continue editing as I think of more. @luisquintanilla

Author:	shaltielshmid
Assignees:	-
Labels:	`api-suggestion`, `area-System.Numerics.Tensors`, `untriaged`
Milestone:	-

[tannergooding]

There is a plan to add the full set of BLAS (level 1, 2, and 3) APIs; which would include basic matrix operations.

Due to the sheer scope of the work, a lot of this is incremental and will be added incrementally to ensure the best experience and API shape can be provided.