Tensor<T> and supporting types

[tannergooding]

Rationale

.NET provides a broad range of support for various development domains, ranging from the creation of performance-oriented framework code to the rapid development of cloud native services, and beyond.

In recent years, especially with the rise of AI and machine learning, there has been a prevalent push towards improving numerical support and allowing developers to more easily write and consume general purpose and reusable algorithms that work with a range of types and scenarios. While .NET's support for scalar algorithms and fixed-sized vectors/matrices is strong and continues to grow, its built-in library support for other concepts, such as tensors and arbitrary length vectors/matrices, could benefit from additional improvements. Today, developers writing .NET applications, services, and libraries currently may need to seek external dependencies in order to utilize functionality that is considered core or built-in to other ecosystems. In particular, for developers incorporating AI and copilots into their existing .NET applications and services, we strive to ensure that the core numerics support necessary to be successful is available and efficient, and that .NET developers are not forced to seek out non-.NET solutions in order for their .NET projects to be successful.

API Proposal

This is extracted from https://github.com/dotnet/designs/pull/316 and the design doc will be updated based on the API review here.

Native Index and Range Types

namespace System.Buffers;

public readonly struct NIndex : IEquatable<NIndex>
{
    public NIndex(nint value, bool fromEnd = false);
    public NIndex(Index value);

    public static NIndex End { get; }
    public static NIndex Start { get; }

    public bool IsFromEnd { get; }
    public nint Value { get; }

    public static implicit operator NIndex(nint value);
    public static implicit operator NIndex(Index value);

    public static explicit operator Index(NIndex value);
    public static explicit operator checked Index(NIndex value);

    public static NIndex FromEnd(nint value);
    public static NIndex FromStart(nint value);

    public static Index ToIndex();
    public static Index ToIndexUnchecked();

    public nint GetOffset(nint length);

    // IEquatable<NIndex>
    public bool Equals(NIndex other);
}

public readonly struct NRange : IEquatable<NRange>
{
    public NRange(NIndex start, NIndex end);
    public NRange(Range value);

    public static NRange All { get; }

    public NIndex End { get; }
    public NIndex Start { get; }

    public static explicit operator Range(NRange value);
    public static explicit operator checked Range(NRange value);

    public static NRange EndAt(NIndex end);
    public (nint Offset, nint Length) GetOffsetAndLength(nint length);
    public static NRange StartAt(NIndex start);

    public static Range ToRange();
    public static Range ToRangeUnchecked();

    // IEquatable<NRange>
    public bool Equals(NRange other);
}

Multi-dimensional Span Types

namespace System.Numerics.Tensors;

public ref struct TensorSpan<T>
{
    public TensorSpan(void* data, nint dataLength, ReadOnlySpan<nint> lengths);
    public TensorSpan(void* data, nint dataLength, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    public TensorSpan(Span<T> span);
    public TensorSpan(Span<T> span, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    public TensorSpan(T[]? array);
    public TensorSpan(T[]? array, int start, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
    public TensorSpan(T[]? array, Index startIndex, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    // Should start just be `nint` and represent the linear index?
    public TensorSpan(Array? array);
    public TensorSpan(Array? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
    public TensorSpan(Array? array, ReadOnlySpan<Index> startIndex, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    // Should we have variants for common ranks (consider 2-5):
    //   public TensorSpan(T[,]? array);
    //   public TensorSpan(T[,]? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
    //   public TensorSpan(T[,]? array, ReadOnlySpan<Index> startIndex, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    public static TensorSpan<T> Empty { get; }

    public bool IsEmpty { get; }
    public nint FlattenedLength { get; }
    public int Rank { get; }
    public ReadOnlySpan<nint> Lengths { get; }
    public ReadOnlySpan<nint> Strides { get; }

    public ref T this[params scoped ReadOnlySpan<nint> indexes] { get; }
    public ref T this[params scoped ReadOnlySpan<NIndex> indexes] { get; }
    public TensorSpan<T> this[params scoped ReadOnlySpan<NRange> ranges] { get; set; }

    // These work like Span<T>, comparing the byref, lengths, and strides, not element-wise
    public static bool operator ==(TensorSpan<T> left, TensorSpan<T> right);
    public static bool operator !=(TensorSpan<T> left, TensorSpan<T> right);

    public static explicit operator TensorSpan<T>(Array? array);
    public static implicit operator TensorSpan<T>(T[]? array);

    // Should we have variants for common ranks (consider 2-5):
    //   public static implicit operator TensorSpan<T>(T[,]? array);

    public static implicit operator ReadOnlyTensorSpan<T>(TensorSpan<T> span);

    public void Clear();
    public void CopyTo(TensorSpan<T> destination);
    public void Fill(T value);
    public void FlattenTo(Span<T> destination);
    public Enumerator GetEnumerator();
    public ref T GetPinnableReference();
    public TensorSpan<T> Slice(params ReadOnlySpan<NIndex> indexes);
    public TensorSpan<T> Slice(params ReadOnlySpan<NRange> ranges);
    public bool TryCopyTo(TensorSpan<T> destination);
    public bool TryFlattenTo(Span<T> destination);

    [ObsoleteAttribute("Equals() on TensorSpan will always throw an exception. Use the equality operator instead.")]
    public override bool Equals(object? obj);

    [ObsoleteAttribute("GetHashCode() on TensorSpan will always throw an exception.")]
    public override int GetHashCode();

    // Do we want `Array ToArray()` to mirror Span<T>?

    public ref struct Enumerator
    {
        public ref readonly T Current { get; }
        public bool MoveNext();
    }
}

public ref struct ReadOnlyTensorSpan<T>
{
    public ReadOnlyTensorSpan(void* data, nint dataLength, ReadOnlySpan<nint> lengths);
    public ReadOnlyTensorSpan(void* data, nint dataLength, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    public ReadOnlyTensorSpan(T[]? array);
    public ReadOnlyTensorSpan(T[]? array, int start, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
    public ReadOnlyTensorSpan(T[]? array, Index startIndex, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    // Should start just be `nint` and represent the linear index?
    public ReadOnlyTensorSpan(Array? array);
    public ReadOnlyTensorSpan(Array? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
    public ReadOnlyTensorSpan(Array? array, ReadOnlySpan<Index> startIndices, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    // Should we have variants for common ranks (consider 2-5):
    //   public ReadOnlyTensorSpan(T[,]? array);
    //   public ReadOnlyTensorSpan(T[,]? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
    //   public ReadOnlyTensorSpan(T[,]? array, ReadOnlySpan<Index> startIndices, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    public static ReadOnlyTensorSpan<T> Empty { get; }

    public bool IsEmpty { get; }
    public nint Length { get; }
    public int Rank { get; }
    public ReadOnlySpan<nint> Lengths { get; }
    public ReadOnlySpan<nint> Strides { get; }

    public ref readonly T this[params scoped ReadOnlySpan<nint> indexes] { get; }
    public ref readonly T this[params scoped ReadOnlySpan<NIndex> indexes] { get; }
    public ReadOnlyTensorSpan<T> this[params scoped ReadOnlySpan<NRange> ranges] { get; set; }

    // These work like ReadOnlySpan<T>, comparing the byref, lengths, and strides, not element-wise
    public static bool operator ==(ReadOnlyTensorSpan<T> left, ReadOnlyTensorSpan<T> right);
    public static bool operator !=(ReadOnlyTensorSpan<T> left, ReadOnlyTensorSpan<T> right);

    public static explicit operator ReadOnlyTensorSpan<T>(Array? array);
    public static implicit operator ReadOnlyTensorSpan<T>(T[]? array);

    // Should we have variants for common ranks (consider 2-5):
    //   public static implicit operator ReadOnlyTensorSpan<T>(T[,]? array);

    public void CopyTo(Span<T> destination);
    public void FlattenTo(Span<T> destination);
    public Enumerator GetEnumerator();
    public ref readonly T GetPinnableReference();
    public ReadOnlyTensorSpan<T> Slice(params ReadOnlySpan<NIndex> indexes);
    public ReadOnlyTensorSpan<T> Slice(params ReadOnlySpan<NRange> ranges);
    public bool TryCopyTo(TensorSpan<T> destination);
    public bool TryFlattenTo(Span<T> destination);

    [ObsoleteAttribute("Equals() on ReadOnlyTensorSpan will always throw an exception. Use the equality operator instead.")]
    public override bool Equals(object? obj);

    [ObsoleteAttribute("GetHashCode() on ReadOnlyTensorSpan will always throw an exception.")]
    public override int GetHashCode();

    // Do we want `Array ToArray()` to mirror Span<T>?

    public ref struct Enumerator
    {
        public ref readonly T Current { get; }
        public bool MoveNext();
    }
}

Tensor Type

public interface IReadOnlyTensor<TSelf, T> : IEnumerable<T>
    where TSelf : IReadOnlyTensor<TSelf, T>
{
    // Should there be APIs for creating from an array, TensorSpan, etc

    static abstract TSelf Empty { get; }

    bool IsEmpty { get; }
    bool IsPinned { get; }
    nint Length { get; }
    int Rank { get; }

    T this[params ReadOnlySpan<nint> indexes] { get; }
    T this[params ReadOnlySpan<NIndex> indexes] { get; }
    TSelf this[params scoped ReadOnlySpan<NRange> ranges] { get; }

    ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan();
    ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan(params scoped ReadOnlySpan<nint> start);
    ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan(params scoped ReadOnlySpan<NIndex> startIndex);
    ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan(params scoped ReadOnlySpan<NRange> ranges);

    void CopyTo(TensorSpan<T> destination);
    void FlattenTo(TensorSpan<T> destination);

    // These are not properties so that structs can implement the interface without allocating:
    void GetLengths(Span<nint> destination);
    void GetStrides(Span<nint> destination);

    ref readonly T GetPinnableReference();
    TSelf Slice(params scoped ReadOnlySpan<nint> start);
    TSelf Slice(params scoped ReadOnlySpan<NIndex> startIndices);
    TSelf Slice(params scoped ReadOnlySpan<NRange> ranges);
    bool TryCopyTo(TensorSpan<T> destination);
    bool TryFlattenTo(TensorSpan<T> destination);
}

public interface ITensor<TSelf, T> : IReadOnlyTensor<TSelf, T>
    where TSelf : ITensor<TSelf, T>
{
    static TSelf Create(ReadOnlySpan<nint> lengths, bool pinned = false);
    static TSelf Create(ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides, bool pinned = false);

    static TSelf CreateUninitialized(ReadOnlySpan<nint> lengths, bool pinned = false);
    static TSelf CreateUninitialized(ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides, bool pinned = false);

    bool IsReadOnly { get; }

    new T this[params ReadOnlySpan<nint> indexes] { get; set; }
    new T this[params ReadOnlySpan<NIndex> indexes] { get; set; }
    new TSelf this[params scoped ReadOnlySpan<NRange> ranges] { get; set; }

    TensorSpan<T> AsTensorSpan();
    TensorSpan<T> AsTensorSpan(params scoped ReadOnlySpan<nint> start);
    TensorSpan<T> AsTensorSpan(params scoped ReadOnlySpan<NIndex> startIndex);
    TensorSpan<T> AsTensorSpan(params scoped ReadOnlySpan<NRange> ranges);

    void Clear();
    void Fill(T value);
    new ref T GetPinnableReference();
}

public sealed class Tensor<T> : ITensor<Tensor<T>, T>
{
    static TSelf ITensor<T>.Create(ReadOnlySpan<nint> lengths, bool pinned = false);
    static TSelf ITensor<T>.Create(ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides, bool pinned = false);

    static TSelf ITensor<T>.CreateUninitialized(ReadOnlySpan<nint> lengths, bool pinned = false);
    static TSelf ITensor<T>.CreateUninitialized(ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides, bool pinned = false);

    public static Tensor<T> Empty { get; }

    public bool IsEmpty { get; }
    public bool IsPinned { get; }
    public nint Length { get; }
    public int Rank { get; }
    public ReadOnlySpan<nint> Lengths { get; }
    public ReadOnlySpan<nint> Strides { get; }

    bool ITensor<T>.IsReadOnly { get; }

    public ref T this[params ReadOnlySpan<nint> indexes] { get; }
    public ref T this[params ReadOnlySpan<NIndex> indexes] { get; }
    public Tensor<T> this[params ReadOnlySpan<NRange> ranges] { get; set; }

    T IReadOnlyTensor<T>.this[params ReadOnlySpan<nint> indexes] { get; }
    T IReadOnlyTensor<T>.this[params ReadOnlySpan<NIndex> indexes] { get; }
    Tensor<T> IReadOnlyTensor<T>.this[params ReadOnlySpan<NRange> ranges] { get; set; }

    public static implicit operator TensorSpan<T>(Tensor<T> value);
    public static implicit operator ReadOnlyTensorSpan<T>(Tensor<T> value);

    public TensorSpan<T> AsTensorSpan();
    public TensorSpan<T> AsTensorSpan(params ReadOnlySpan<nint> start);
    public TensorSpan<T> AsTensorSpan(params ReadOnlySpan<NIndex> startIndex);
    public TensorSpan<T> AsTensorSpan(params ReadOnlySpan<NRange> ranges);

    public ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan();
    public ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan(params ReadOnlySpan<nint> start);
    public ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan(params ReadOnlySpan<NIndex> startIndex);
    public ReadOnlyTensorSpan<T> AsReadOnlyTensorSpan(params ReadOnlySpan<NRange> ranges);

    public void Clear();
    public void CopyTo(TensorSpan<T> destination);
    public void FlattenTo(TensorSpan<T> destination);
    public void Fill(T value);
    public Enumerator GetEnumerator();
    public ref T GetPinnableReference();
    public Tensor<T> Slice(params ReadOnlySpan<nint> start);
    public Tensor<T> Slice(params ReadOnlySpan<NIndex> startIndices);
    public Tensor<T> Slice(params ReadOnlySpan<NRange> ranges);
    public bool TryCopyTo(TensorSpan<T> destination);
    public bool TryFlattenTo(TensorSpan<T> destination);

    void IReadOnlyTensor<T>.GetLengths(Span<nint> destination);
    void IReadOnlyTensor<T>.GetStrides(Span<nint> destination);
    ref readonly T IReadOnlyTensor<T>.GetPinnableReference();

    // The behavior of Equals, GetHashCode, and ToString needs to be determined

    // For ToString, is the following sufficient to help mitigate potential issues:
    //   public string ToString(params ReadOnlySpan<nint> maximumLengths);

    public ref struct Enumerator
    {
        public ref readonly T Current { get; }
        public bool MoveNext();
    }
}

public static partial class Tensor
{
    public static TSelf Create(ReadOnlySpan<nint> lengths, bool pinned = false);
    public static TSelf Create(ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides, bool pinned = false);

    public static TSelf CreateUninitialized(ReadOnlySpan<nint> lengths, bool pinned = false);
    public static TSelf CreateUninitialized(ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides, bool pinned = false);

    // Effectively mirror the TensorPrimitives surface area. Following the general pattern
    // where we will return a new tensor and an overload that takes in the destination explicitly.
    //   public static Tensor<T> Add<T>(Tensor<T> left, Tensor<T> right)
    //       where T : IAdditionOperators<T, T, T>;
    //   public static void Add<T>(TensorSpan<T> left, ReadOnlyTensorSpan<T> right)
    //       where T : IAdditionOperators<T, T, T>;
    //   public static TTensor Add<TTensor, T>(TensorSpan<T> left, ReadOnlyTensorSpan<T> right)
    //       where T : IAdditionOperators<T, T, T>
    //       where TTensor : ITensor<TTensor, T>;

    // Consider whether we should have one named `*InPlace`, for easier chaining, such as:
    //   public static Span<T> AddInPlace<T>(TensorSpan<T> left, ReadOnlyTensorSpan<T> right)
    //       where T : IAdditionOperators<T, T, T>;

    // The language ideally has extension operators such that we can define `operator +` instead of `Add`
    // Without this support, we end up having to have `TensorNumber<T>`, `TensorBinaryInteger<T>`, etc
    // as we otherwise cannot correctly expose the operators based on what `T` supports

    // APIs that would return `bool` like `GreaterThan` are split into 3. Following the general
    // pattern already established for our SIMD vector types.
    // * public static ITensor<T> GreaterThan(ITensor<T> x, ITensor<T> y);
    // * public static bool GreaterThanAny<T>(ITensor<T> x, ITensor<T> y);
    // * public static bool GreaterThanAll<T>(ITensor<T> x, ITensor<T> y);
}

Additional Notes

We know we need to support having the backing memory allocated in native so that large array allocations can work. Our current thinking is that this will be a distinct/separate type (possibly named NativeTensor<T>), but are still finalizing the details on how lifetimes and ownership will work correctly.

By default, operations will allocate and return a Tensor<T> from operations. As part of the above support for NativeTensor<T> we are looking at ways the users can override this default behavior, it will likely come with some ThreadStatic member that allows users to provide a custom allocator.

[bartonjs]

Video

• Added NativeIndex..ctor(Index), NativeIndex.ToIndex, and similar to NativeRange.
• Added a Span-accepting overload to SpanND to mirror the Array one, and similar to ReadOnlySpanND
• We're not sure if System is the right namespace for NativeIndex and NativeRange, but want more people in the room to discuss.
• The name SpanND did not meet with universal love and harmony, needs more people to discuss.
• The namespaces are also something to further discuss
• We added indexers on SpanND/ROSpanND to accept params NativeIndex
• We added (Try)FlattenTo to copy a SpanND into a Span.

namespace System;

public readonly struct NativeIndex : IEquatable<NativeIndex>
{
    public NativeIndex(nint value, bool fromEnd = false);
    public NativeIndex(Index value);

    public static NativeIndex End { get; }
    public static NativeIndex Start { get; }

    public bool IsFromEnd { get; }
    public nint Value { get; }

    public static implicit operator NativeIndex(nint value);
    public static implicit operator NativeIndex(Index value);

    public static explicit operator Index(NativeIndex value);
    public static explicit operator checked Index(NativeIndex value);

    public Index ToIndex();

    public static NativeIndex FromEnd(nint value);
    public static NativeIndex FromStart(nint value);

    public nint GetOffset(nint length);
}

public readonly struct NativeRange : IEquatable<NativeRange>
{
    public NativeRange(NativeIndex start, NativeIndex end);
    public NativeRange(Range range);

    public static NativeRange All { get; }

    public NativeIndex End { get; }
    public NativeIndex Start { get; }

    public static explicit operator Range(NativeRange value);
    public static explicit operator checked Range(NativeRange value);

    public Range ToRange();

    public static NativeRange EndAt(NativeIndex end);
    public (nint Offset, nint Length) GetOffsetAndLength(nint length);
    public static NativeRange StartAt(NativeIndex start);
}

namespace System;

public readonly ref struct SpanND<T>
{
    public SpanND(void* pointer, ReadOnlySpan<nint> lengths);
    public SpanND(void* pointer, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    public SpanND(Span<T> span);
    public SpanND(Span<T> span, ReadOnlySpan<nint> offsets, ReadOnlySpan<nint> lengths);

    public SpanND(Array? array);
    public SpanND(Array? array, ReadOnlySpan<nint> offsets, ReadOnlySpan<nint> lengths);

    // Do we want overloads for common dimensions like `T[,]` and `T[,,]`?

    public static SpanND<T> Empty { get; }

    public bool IsEmpty { get; }
    public ReadOnlySpan<nint> Lengths { get; }
    public nint LinearLength { get; }
    public int Rank { get; }
    public ReadOnlySpan<nint> Strides { get; }

    public ref T this[params ReadOnlySpan<nint> indices] { get; }
    public ref T this[params ReadOnlySpan<NativeIndex> indices] { get; }

    public static bool operator ==(SpanND<T> left, SpanND<T> right);
    public static bool operator !=(SpanND<T> left, SpanND<T> right);

    public static implicit operator SpanND<T>(Array array);
    public static implicit operator SpanND<T>(Span<T> span);
    public static implicit operator ReadOnlySpanND<T>(SpanND<T> span);

    public void Clear();

    public void CopyTo(SpanND<T> destination);
    public bool TryCopyTo(SpanND<T> destination);

    public void FlattenTo(Span<T> destination);
    public bool TryFlattenTo(Span<T> destination);

    public void Fill(T value);
    public Enumerator GetEnumerator();
    public ref T GetPinnableReference();
    public SpanND<T> Slice(params ReadOnlySpan<NativeIndex> indices);
    public SpanND<T> Slice(params ReadOnlySpan<NativeRange> ranges);

    // Do we want `Array ToArray()` to mirror Span<T>?

    public ref struct Enumerator
    {
        public ref readonly T Current { get; }
        public bool MoveNext();
    }
}

public readonly ref struct ReadOnlySpanND<T>
{
    public ReadOnlySpanND(void* pointer, ReadOnlySpan<nint> lengths);
    public ReadOnlySpanND(void* pointer, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);

    public ReadOnlySpanND(ReadOnlySpan<T> span);
    public ReadOnlySpanND(ReadOnlySpan<T> span, ReadOnlySpan<nint> starts, ReadOnlySpan<nint> lengths);

    public ReadOnlySpanND(Array? array);
    public ReadOnlySpanND(Array? array, ReadOnlySpan<nint> starts, ReadOnlySpan<nint> lengths);

    // Do we want overloads for common dimensions like `T[,]` and `T[,,]`?

    public static ReadOnlySpanND<T> Empty { get; }

    public bool IsEmpty { get; }
    public ReadOnlySpan<nint> Lengths { get; }
    public nint LinearLength { get; }
    public int Rank { get; }
    public ReadOnlySpan<nint> Strides { get; }

    public ref readonly T this[params ReadOnlySpan<nint> indices] { get; }
    public ref readonly T this[params ReadOnlySpan<NativeIndex> indices] { get; }

    public static bool operator ==(ReadOnlySpanND<T> left, ReadOnlySpanND<T> right);
    public static bool operator !=(ReadOnlySpanND<T> left, ReadOnlySpanND<T> right);

    public static implicit operator ReadOnlySpanND<T>(Array array);

    public void CopyTo(SpanND<T> destination);
    public bool TryCopyTo(SpanND<T> destination);

    public void FlattenTo(Span<T> destination);
    public bool TryFlattenTo(Span<T> destination);

    public Enumerator GetEnumerator();
    public ref readonly T GetPinnableReference();
    public ReadOnlySpanND<T> Slice(params ReadOnlySpan<NativeIndex> indices);
    public ReadOnlySpanND<T> Slice(params ReadOnlySpan<NativeRange> ranges);

    // Do we want `Array ToArray()` to mirror Span<T>?

    public ref struct Enumerator
    {
        public ref readonly T Current { get; }
        public bool MoveNext();
    }
}

[Clockwork-Muse]

... Isn't the ability to return buffers backed by a native buffer part of the point of MemoryManager?

[Neme12]

I really, really don't think we should expose NativeIndex and NativeRange (and potentially NativeSpan for that matter). Instead, the existing types should be updated to accept nint and also to be able to return nint using the new language feature that is coming of BinaryCompatOnly/overload resolution preference, which should allow overloading by return type. This would be a source breaking change, but not a binary breaking change since existing binaries would call the original methods. Otherwise, it will be really weird why there's Span and NativeSpan but no NativeSpanND

[Neme12]

If we're going with abbreviations in the name (i.e. not something like SpanMultiDimensional), I would definitely go with SpanMD over SpanND. I can immediately understand what the MD in SpanMD means, but I would have no idea what a SpanND is - I would have to look at the documentation to understand that it's a multi-dimensional span... and even after I look at the documentation and learn that, I would still have no idea what the ND stands for (and I would probably think the N stands for native, so native something), whereas SpanMD is immediately clear.

[Neme12]

As far as the namespace goes, System.Numerics might be better than System (for SpanMD, not for NativeIndex & NativeRange), since I imagine this will be a lot less common than regular Span, and only used in specialized domains related to numerics.

[Neme12]

public SpanND(Span<T> span, ReadOnlySpan<nint> offsets, ReadOnlySpan<nint> lengths);

How would this constructor work? It mirrors the Array one, but the array one calculates the strides based on the array, and there are no strides here. How are the strides determined here? Or is this more like the void* one where the span is from the first element, as opposed to the whole thing we're slicing? It's really unclear.

[Neme12]

I would also add the equivalent of the void* constructor but with a ref instead of a pointer. But that would be more appropriate on MemoryMarshal next to CreateSpan, so:

namespace System.Runtime.InteropServices;

public static class MemoryMarshal
{
    // Existing method:
    public static Span<T> CreateSpan<T>(ref T reference, int length);

    // New methods:
    public static SpanND<T> CreateSpanND<T>(ref T reference, ReadOnlySpan<nint> lengths);
    public static SpanND<T> CreateSpanND<T>(ref T reference, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
}

(I'll start putting everything in one comment so that people don't have many pings).

---

Also, why do the void* constructors take void* instead of T*?

---

And is NativeRange missing an implicit conversion from Range, similar to how NativeIndex has an implicit conversion from Index? And does NativeIndex need an implicit conversion from int, or will it "just work" and the int will be converted to nint and then the implicit conversion from nint will be called?

I would also expect the ToIndex and ToRange methods to be checked and throw if it cannot fit, I hope that will be the case.

Note: In the API review, Jeremy mentioned that there's a guideline that there should be named methods in addition to (or instead of) conversion operators because operators aren't discoverable. This isn't really the case anymore, IntelliSense now shows operators: And when you hit tab, it conveniently replaces i. with ((int)i). I'm just mentioning that since it might change the calculus of when named methods should be added in addition to operators, although named methods might still be nice because it reads a little nicer, especially when you want the conversion to be checked and you'd have to surround the cast with a checked expression.

[tannergooding]

Instead, the existing types should be updated to accept nint and also to be able to return nint using the new language feature that is coming of BinaryCompatOnly/overload resolution preference, which should allow overloading by return type. This would be a source breaking change, but not a binary breaking change since existing binaries would call the original methods. Otherwise, it will be really weird why there's Span and NativeSpan but no NativeSpanND

This is not going to happen. It has been suggested and discussed in depth for years and it is "too breaking". There is simply too much existing code that assumes the lengths can only be int in length and it would lead to subtle bugs, corner cases, and a non-trivial perf pessimization to almost all existing code.

[tannergooding]

A lot of the other feedback given was already covered in API review and the proposal will be updated accordingly before it goes back to review.

Most of the questions of "why" can be answered by prior precedent, convenience, or it being historical artifacts based on the point in time the prior precedent was implemented. .NET then tries to stay overall consistent unless there is sufficient reason to deviate.

For example, we use void* because T* used to be completely illegal. It was then relaxed only for where T : unmanaged and so wouldn't have been usable for some Span<T> constructor given T is unconstrained. It was then relaxed again even more recently to allow any T and to warn if T was unconstrained. void* avoids the need to suppress the diagnostic and makes several other concepts simpler, so it ends up being more than sufficient and overall consistent with prior art.

Cases where we have opted to explicitly deviate include places that don't have more widespread prior art, like Array using LongLength and GetLength and it being well known that these are historical points of confusion for users that lead them into potential pits of failure and where slightly different names and accounting for modern features gives us something overall better.

[neon-sunset]

Somewhat unrelated but is there a reason for SpanND over NDSpan (like numpy's ndarray)? 😄

[jubruckne]

What about HyperSpan since it's a view into a hypercube, basically...

NativeIndex / NativeRange... I feel that "Native" implies to me that it's doing something with unmanaged memory (as NativeTensor is envisioned to do).

As for FlattenTo... What is this method envisioned to be used for? If it's for interop or persistence, it would probably need a parameter to specify the target layout (row/column major).

[tannergooding]

Somewhat unrelated but is there a reason for SpanND over NDSpan (like numpy's ndarray)?

Also covered in the review. The general premise is that the natural name for a type of known size would be Span2D, Span3D, etc.. It then becomes consistent to use SpanND to represent a type of arbitrary dimensionality. N was picked because that is the common way to refer to this data, it is n-dimensional, and many people preferred it over MD (hence also why languages like python have choosen ndarray, as nd is a very common term used in the domain space).

NativeIndex / NativeRange... I feel that "Native" implies to me that it's doing something with unmanaged memory (as NativeTensor is envisioned to do).

Native has many different meanings in many different contexts, but historically in .NET it has meant platform-sized. For example, nint is named such because it is a native integer which corresponds to the official IL name for the type going back to .NET Framework v1.0. Unmanaged is typically the term used to mean things that are done in native code or which are compatible with native code or otherwise manually managed/handled, as they are not managed by the GC or runtime.

What is this method envisioned to be used for? If it's for interop or persistence, it would probably need a parameter to specify the target layout (row/column major).

It is a common API for libraries providing such functionality and is simply meant to get a linear view of the data via a copy operation (as compared to a reshape operation that may make data access effectively random).

You cannot trivially specify target layout via parameterization as it is not as simple as "row vs column major", especially as you get to higher dimensionality. As per API review, this would be done via the explicit Reshape APIs that are will be covered in a future API review as we knew we couldn't get to everything in 2 hours. Thus, you would tensor.Reshape(...).FlattenTo(...) (or another API like Transpose if you wanted one of the well defined Reshape operations) if you needed it stored in a particular format.

Tensors themselves will stored "row-major" as that is how .NET is explicitly documented to work and how most other languages/runtimes (and physical memory) actually work. Which is to say given an index [a, b, c, d]. It is a-major and d-minor, effectively ordered from the largest factor first to the smallest factor of the index last. Rehsape operations will allow you to create alternative views without copying such that d may not have a stride of 1 and this will allow you to, for example, get a column-major view of a row-major stored dataset without copy, but whether that is better or worse is algorithm dependent since random memory access may be worse overall for performance.

[Neme12]

If there was an implicit conversion from an index to a range, I don't think it should create a range of length one, but instead a range from the specified index to the end, because that's how span's Slice behaves when you only pass an index and not a range.

[DaZombieKiller]

To throw a name suggestion out there, one that comes to mind is RankSpan<T>. Rank is much shorter than Dimension/Dimensional while still carrying the same meaning and not being an abbreviation.

---

public SpanND<T> Slice(params ReadOnlySpan<NativeIndex> indices);
public SpanND<T> Slice(params ReadOnlySpan<NativeRange> ranges);

Should there be an additional params ReadOnlySpan<nint> overload of Slice for parity with the indexer?

---

// Do we want `Array ToArray()` to mirror Span<T>?

Judging by the return type, I assume this would allocate a multi-dimensional array? Would it make sense to also have T[] ToFlattenedArray() or T[] ToLinearArray()?

[frankbuckley]

• The name SpanND did not meet with universal love and harmony, needs more people to discuss.

Why not TensorSpan<T> and ReadOnlyTensorSpan<T> as in the design doc?

(Though MatrixSpan<T> might be preferable to Span2D<T>)

[colejohnson66]

I'm partial to [ReadOnly]SpanND simply because I'd expect I could use them for normal access of a 2D array, not just tensors/matrices. However, while I understand the intent behind the name (a span for an "n" dimensional array) and how it would follow a convention ("span 2D" -> "span ND"), I'm in agreement with the API review comment: the name isn't good, but not necessarily for the same reasons. NDSpan feels more natural at first glance. While #DSpan type names wouldn't be allowed, NDSpan would.

Ultimately, while I agree with @tannergooding that the name should be short, "ND" is too short and unclear IMO. I'd suggest either [ReadOnly]NDimSpan or [ReadOnly]SpanNDim. That would clarify their purpose a bit better.

[tannergooding]

To throw a name suggestion out there, one that comes to mind is RankSpan

RankSpan is too confusing imo. It requires users to be familiar with the term rank, which I'd guess is even less familiar to users and may be confused for other terminology. Additionally, users may think that it is effectively similar to a custom type Span<Rank>.

Why not TensorSpan and ReadOnlyTensorSpan as in the

We determined these types should not be restricted to only Tensor<T> and they should be general purpose span types.

(Though MatrixSpan might be preferable to Span2D)

MatrixSpan has similar issues as detailed above. Many users may assume it is a specialized form of Span<Matrix> and that itself may lead to assumptions around dimensionality as you typically consider it scalar (0D), vector (1D), matrix (2D), ... where Tensor is the domain specific term encompassing all of them

---

It's worth explicitly noting that one of the reasons that ND was preferred by almost everyone that took an initial look is that its the almost ubiquitous terminology used in this domain and is the common term used in other languages (like python and ndarray).

This isn't some arbitrary thing that .NET would be opting to do that differs from the rest of the broader ecosystem, it would only differ initially from the more standard .NET convention. However, .NET has often used abbreviations where it does make sense, especially for more core or primitive types (it is Int32 not Integer32) and there are places we've been more verbose that people have voiced opinions around (ReadOnlySpan could have probably been ROSpan given its prevalence).

I disagree that ND is too unclear and rather that its simply a term one needs to learn when working in the broader context of ML, AI, Intelligent Apps, and the types of programming languages, samples, etc that get used in that space. I expect that not using ND will cause overall more long term confusion, especially for people who are primarily .NET developers but who want to interact with other prominent libraries (TorchSharp, TensorFlow.NET, ONNYX, etc) and who may be following samples from other languages (particularly python/numpy). -- That is to say, us not using nd doesn't fix the problem of people not familiar with the domain space needing to learn what nd means, it is going to be front and center regardless.

[bartonjs]

Video

• After a very long debate on names, we ended up with NativeIndex => NIndex and NativeRange => NRange
• After a similarly long debate, we think SpanND => TensorSpan
• NIndex/NRange go to System.Buffers
• NIndex/NRange are probably now considered complete
• For pairing To-prefixed methods with explicit conversions, the type-only name (ToIndex) means checked, and the alternative named one gets the suffix -Unchecked (ToIndexUnchecked)
• TensorSpan probably goes to System.Numerics.Tensors
• TensorSpan did not get reviewed today other than its name and namespace, so this issue will need a round 3.

namespace System.Buffers;

public readonly struct NIndex : IEquatable<NIndex>
{
    public NIndex(nint value, bool fromEnd = false);
    public NIndex(Index value);

    public static NIndex End { get; }
    public static NIndex Start { get; }

    public bool IsFromEnd { get; }
    public nint Value { get; }

    public static implicit operator NIndex(nint value);
    public static implicit operator NIndex(Index value);

    public static explicit operator Index(NIndex value);
    public static explicit operator checked Index(NIndex value);

    public static NIndex FromEnd(nint value);
    public static NIndex FromStart(nint value);

    // Should this always do checked or unchecked, how do users get the other?
    public static Index ToIndex();
    public static Index ToIndexUnchecked();

    public nint GetOffset(nint length);

    // IEquatable<NIndex>
    public bool Equals(NIndex other);
}

public readonly struct NRange : IEquatable<NRange>
{
    public NRange(NIndex start, NIndex end);
    public NRange(Range value);

    public static NRange All { get; }

    public NIndex End { get; }
    public NIndex Start { get; }

    public static explicit operator Range(NRange value);
    public static explicit operator checked Range(NRange value);

    public static NRange EndAt(NIndex end);
    public (nint Offset, nint Length) GetOffsetAndLength(nint length);
    public static NRange StartAt(NIndex start);

    // Should this always do checked or unchecked, how do users get the other?
    public static Range ToRange();
    public static Range ToRangeUnchecked();

    // IEquatable<NRange>
    public bool Equals(NRange other);
}

namespace System.Numerics.Tensors;

public ref struct TensorSpan<T>
{
    // Should this be in the shape of the following instead:
    //   public SpanND(void* pointer, nint length, ReadOnlySpan<nint> shape);
    //   public SpanND(void* pointer, nint length, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    public SpanND(void* pointer, ReadOnlySpan<nint> shape);
    public SpanND(void* pointer, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    public SpanND(Span<T> span);
    public SpanND(Span<T> span, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    public SpanND(T[]? array);
    public SpanND(T[]? array, int start, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);
    public SpanND(T[]? array, Index startIndex, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    // Should start just be `nint` and represent the linear index?
    public SpanND(Array? array);
    public SpanND(Array? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);
    public SpanND(Array? array, ReadOnlySpan<Index> startIndex, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    // Should we have variants for common ranks (consider 2-5):
    //   public SpanND(T[,]? array);
    //   public SpanND(T[,]? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);
    //   public SpanND(T[,]? array, ReadOnlySpan<Index> startIndex, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    public static SpanND<T> Empty { get; }

    public bool IsEmpty { get; }
    public nint Length { get; }
    public int Rank { get; }
    public ReadOnlySpan<nint> Shape { get; }
    public ReadOnlySpan<nint> Strides { get; }

    public ref T this[params scoped ReadOnlySpan<nint> indices] { get; }
    public ref T this[params scoped ReadOnlySpan<NativeIndex> indices] { get; }
    public SpanND<T> this[params scoped ReadOnlySpan<NativeRange> ranges] { get; set; }

    // These work like Span<T>, comparing the byref, shape, and strides, not element-wise
    public static bool operator ==(SpanND<T> left, SpanND<T> right);
    public static bool operator !=(SpanND<T> left, SpanND<T> right);

    public static explicit operator SpanND<T>(Array? array);
    public static implicit operator SpanND<T>(T[]? array);

    // Should we have variants for common ranks (consider 2-5):
    //   public static implicit operator SpanND<T>(T[,]? array);

    public static implicit operator ReadOnlySpanND<T>(SpanND<T> span);

    public void Clear();
    public void CopyTo(SpanND<T> destination);
    public void Fill(T value);
    public void FlattenTo(SpanND<T> destination);
    public Enumerator GetEnumerator();
    public ref T GetPinnableReference();
    public SpanND<T> Slice(params ReadOnlySpan<NativeIndex> indices);
    public SpanND<T> Slice(params ReadOnlySpan<NativeRange> ranges);
    public bool TryCopyTo(SpanND<T> destination);
    public bool TryFlattenTo(SpanND<T> destination);

    [ObsoleteAttribute("Equals() on SpanND will always throw an exception. Use the equality operator instead.")]
    public override bool Equals(object? obj);

    [ObsoleteAttribute("GetHashCode() on SpanND will always throw an exception.")]
    public override int GetHashCode();

    // Do we want `Array ToArray()` to mirror Span<T>?

    public ref struct Enumerator
    {
        public ref readonly T Current { get; }
        public bool MoveNext();
    }
}

public ref struct ReadOnlySpanND<T>
{
    // Should this be in the shape of the following instead:
    //   public ReadOnlySpanND(void* pointer, nint length, ReadOnlySpan<nint> shape);
    //   public ReadOnlySpanND(void* pointer, nint length, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    public ReadOnlySpanND(void* pointer, ReadOnlySpan<nint> shape);
    public ReadOnlySpanND(void* pointer, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    public ReadOnlySpanND(T[]? array);
    public ReadOnlySpanND(T[]? array, int start, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);
    public ReadOnlySpanND(T[]? array, Index startIndex, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    // Should start just be `nint` and represent the linear index?
    public ReadOnlySpanND(Array? array);
    public ReadOnlySpanND(Array? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);
    public ReadOnlySpanND(Array? array, ReadOnlySpan<Index> startIndices, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    // Should we have variants for common ranks (consider 2-5):
    //   public ReadOnlySpanND(T[,]? array);
    //   public ReadOnlySpanND(T[,]? array, ReadOnlySpan<int> start, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);
    //   public ReadOnlySpanND(T[,]? array, ReadOnlySpan<Index> startIndices, ReadOnlySpan<nint> shape, ReadOnlySpan<nint> strides);

    public static ReadOnlySpanND<T> Empty { get; }

    public bool IsEmpty { get; }
    public nint Length { get; }
    public int Rank { get; }
    public ReadOnlySpan<nint> Shape { get; }
    public ReadOnlySpan<nint> Strides { get; }

    public ref readonly T this[params scoped ReadOnlySpan<nint> indices] { get; }
    public ref readonly T this[params scoped ReadOnlySpan<NativeIndex> indices] { get; }
    public ReadOnlySpanND<T> this[params scoped ReadOnlySpan<NativeRange> ranges] { get; set; }

    // These work like ReadOnlySpan<T>, comparing the byref, shape, and strides, not element-wise
    public static bool operator ==(ReadOnlySpanND<T> left, ReadOnlySpanND<T> right);
    public static bool operator !=(ReadOnlySpanND<T> left, ReadOnlySpanND<T> right);

    public static explicit operator ReadOnlySpanND<T>(Array? array);
    public static implicit operator ReadOnlySpanND<T>(T[]? array);

    // Should we have variants for common ranks (consider 2-5):
    //   public static implicit operator ReadOnlySpanND<T>(T[,]? array);

    public void CopyTo(ReadOnlySpanND<T> destination);
    public void FlattenTo(ReadOnlySpanND<T> destination);
    public Enumerator GetEnumerator();
    public ref readonly T GetPinnableReference();
    public ReadOnlySpanND<T> Slice(params ReadOnlySpan<NativeIndex> indices);
    public ReadOnlySpanND<T> Slice(params ReadOnlySpan<NativeRange> ranges);
    public bool TryCopyTo(SpanND<T> destination);
    public bool TryFlattenTo(SpanND<T> destination);

    [ObsoleteAttribute("Equals() on ReadOnlySpanND will always throw an exception. Use the equality operator instead.")]
    public override bool Equals(object? obj);

    [ObsoleteAttribute("GetHashCode() on ReadOnlySpanND will always throw an exception.")]
    public override int GetHashCode();

    // Do we want `Array ToArray()` to mirror Span<T>?

    public ref struct Enumerator
    {
        public ref readonly T Current { get; }
        public bool MoveNext();
    }
}

[Neme12]

How do the unchecked conversion variants (ToIndexUnchecked & ToRangeUnchecked) even make sense? In case of overflow, it would go negative? But an Index cannot be negative. It's always checked in the constructors that the value isn't negative, and the Index even relies on that being the case, as it stores "from end" internally being stored as negative (~value). But this is only an implementation detail - nowhere is this exposed in the public API. To me, the unchecked conversions make no sense. I can't think of a single universally accepted and well understood behavior that it could have (other than undefined behavior, that depends on implementation details).

Are they really actually needed? Or did someone just think of them for symmetry with the otherwise checked ones? I also can't think of any other named conversion operators that have unchecked variants - named conversion operators have always been checked in .NET from what I can recall (and user defined conversion operators as well - it's only been since generic math that this has changed). Since there has never been any need for "unchecked" constructors either and everyone's been fine with the default being checked, I don't think there's need for unchecked/unsafe conversions either.

[bartonjs]

I can't think of a single universally accepted and well understood behavior that it could have (other than undefined behavior, that depends on implementation details).

Turning 63-or-31 addressing bits into 31 addressing bits: int newIdx = (int)(idx & 0x7FFF_FFFF); (then or-ing in the sign bit to track from left or from right).

Which is admittedly better (applying domain logic) than the naive int newIdx = (int)idx; in a default/unchecked scope.

[Neme12]

I guess, but that still exposes implementation details - the implementation detail that fromEnd is stored as ~value, since "fromStart" could now turn into "fromEnd" and vice versa. This is never exposed in the public API. For all we know from outside, they could be separate fields. So I would think of this operation as not only "unchecked", but also "unsafe", because of that dependency on implementation details.

I'm not sure adding this would be worth it. There has never been need for unchecked constructor/factory methods either. And if it were added, I would think of it more like "unsafe" rather than "unchecked", following the precedent of APIs that have implementation-defined behavior being suffixed with "unsafe".

[Neme12]

Well, of course this could be said to be well-defined. But from the outside perspective, it still makes little sense that this would be the behavior, unless you happen to know that implementation details. And all this really worth it for avoiding a single branch? That branch has never been an issue in the constructor or even in Index's implicit conversion from int.

[Neme12]

Actually, given that Index's implicit conversion from int is always checked and has no unchecked variant at all, I think it would be better to follow that pattern and make even the implicit conversions also always checked. I would have never expected for a user-defined conversion to be unchecked in .NET, since... they had always been checked pretty much everywhere that I recall. As I said, unchecked variants were only added since generic math, and I think they only make sense on number types. This might seem number-like but really cannot be thought of as a single number, it's an abstract concept that can represent "from end" and that cannot map to a number unless you materialize the Index to a concrete length by using GetOffset.

To me, this seems like a case of applying the new unchecked/checked operators and applying them too broadly. I'm worried this might be a little bit of a footgun, since "fromStart" can now turn into "fromEnd" and vice versa. And it's not like numbers, where negatives can go into positives by addition etc. Here, "fromStart" and "fromEnd" are separate universes. You cannot ever get into "fromEnd" by incremeting an "Index" - it would always stay as "fromStart", since we don't know where the end is.

[tannergooding]

The misunderstanding here is because you're making a bad assumption about how unchecked behaves.

Unchecked conversions typically functions via truncation, which is always well-defined. Truncating from x-bits to y-bits simply involves taking the lowest y-bits and ignoring the others.

The bad assumption you're making is that you're assuming Index has a domain of 32-bits, when that isn't the case. The use of int32 is simply due the fact that's the best way to represent it on a computer, the actual constructor and contract of the type is that it functionally more a uint31 that is stored as an int32 for performance reasons.

There are multiple APIs throughout .NET that function the same way, such as Span<T>.Length, Array.Length, List<T>.Count, and more where while the type returned is a 32-bit signed integer, it is functionally a never negative 32-bit signed integer instead and therefore logically a 31-bit unsigned integer. For some special cases (like array/span) the runtime may even optimize based on this assumption.

Thus, the unchecked conversion of NIndex to Index is not a 64-bit to a 32-bit truncation. It is a 63-bit to a 31-bit truncation of the Value property and a 1-to-1 propagation of the FromEnd property.

[colejohnson66]

Glad you decided against SpanND :) @bartonjs's comment still says [ReadOnly]SpanND though.

Implementation question for [ReadOnly]TensorSpan: what's the behavior of passing null for the Array?-taking constructors. On [ReadOnly]Span, it sets the reference to Unsafe.NullRef<T>() and the length to zero (this = default;). How would that work here with something that is supposed to handle multiple dimensions? Would it do this = default as well? Because that would probably set Shape and Strides to Unsafe.NullRef<T>() as well, which could be unexpected.

[Neme12]

@tannergooding Oh right, thanks for the explanation. I thought what was meant by idx in int newIdx = (int)(idx & 0x7FFF_FFFF); was the private _value field, not the externally visible Value property. That makes a lot more sense behavior-wise, although I'd still question the value of unchecked conversions in this case. My gut feeling is that if this was added before generic math was a thing, everything would have been checked, and only checked, like other user-defined conversions, e.g on BigInteger and IntPtr. Noone ever needed unchecked variants for those either, and they're still only available now from a generic math context for something user-defined like BigInteger. And I've been used to assuming that conversions on user-defined types are always checked, because that's what they've always been historically. The idea that I would cast an NIndex to Index and it would be an unchecked conversion that could result in nonsense data would be a big surprise to me.

[Neme12]

BTW, I would prefer to go with lengths as opposed to shape for the parameter and property names. When I first saw this proposal (as someone without domain knowledge) and I saw this parameter, I could immediately tell what it meant. Looking at it now that it's been renamed to shape, I would have no idea what a "shape" is, or why is a collection of things called a singular word at all - it's not a single thing.

[gaviny82]

The Tensor<T>.Rank property would be ambiguous in linear algebra if we use a second order tensor to represent a matrix, since the rank of a matrix is defined as the number of linearly independent columns. In other languages, Dimension is used to refer to the number of indices needed to access an element in an $n$-dimensional array:

MATLAB

% MATLAB Example:
A = rand(4, 3, 2);  % Create a 3D array
orderOfA = ndims(A);  % This will return 3

NumPy

# NumPy Example:
B = np.random.rand(4, 3, 2)  # Create a 3D array
orderOfB = B.ndim  # This will return 3

PyTorch

# torch.Tensor Example:
C = torch.Tensor(B) # Create a torch.Tensor
orderOfC = C.ndim # This will return 3

However, if Tensor<T>.Dimensions is used, it is still ambiguous if it represents a vector. It always returns $1$, but the dimension of a vector $a\in R^n$ typically refers to the dimension of the vector space $n$, which is equal to the number of elements in the vector. The same applies for matrices $A\in R^{m\times n}$. Would Tensor<T>.Order be a better option?

In terms of naming the span, I still think SpanND<T> is better. TensorSpan<T> sounds like a span of Tensor<T> objects, and it is more confusing when we use it for data structures in lower dimensions. What does it mean by the TensorSpan of a matrix or a vector? SpanND is more clear in the sense of representing a general $n$-dimensional array. It can be used for higher dimensional arrays like int[,,], but TensorSpan sounds specific to tensors. The community toolkit also has an implementation for Span2D<T>. I don't think using N in SpanND is an issue, since it is commonly used in math and engineering for "n-dimensional".

[gaviny82]

BTW, I would prefer to go with lengths as opposed to shape for the parameter and property names. When I first saw this proposal (as someone without domain knowledge) and I saw this parameter, I could immediately tell what it meant. Looking at it now that it's been renamed to shape, I would have no idea what a "shape" is, or why is a collection of things called a singular word at all - it's not a single thing.

Mathematically it's called the dimension of a tensor. For example, the dimension of a matrix $A\in R^{3\times2}$ is $3\times 2 = 6$. We don't say the lengths of a vector/matrix/tensor. I guess Shape is used because it's used in NumPy.

[Neme12]

@gaviny82

In terms of naming the span, I still think SpanND<T> is better. TensorSpan<T> sounds like a span of Tensor<T> objects, and it is more confusing when we use it for data structures in lower dimensions. What does it mean by the TensorSpan of a matrix or a vector? SpanND is more clear in the sense of representing a general n-dimensional array. It can be used for higher dimensional arrays like int[,,], but TensorSpan sounds specific to tensors. The community toolkit also has an implementation for Span2D<T>. I don't think using N in SpanND is an issue, since it is commonly used in math and engineering for "n-dimensional".

This was discussed for a long time in the API review with @tannergooding constantly reiterating this point, but the decision has been made regardless, so I don't think it's worth discussing the names any further. It's true that the span isn't specific to "Tensor" and can represent any multi-dimensional arrays/data, however you can think of any multi-dimensional array as a tensor - if I oversimplify a little bit, I would say that "tensor" is just a mathematical way of saying multi-dimensional array. A programmer might not think of their bitmap 2d array as a tensor nor a matrix, but that's essentially what it is mathematically.

[Neme12]

Now, it's been a long time since I took linear algebra, and from what I understand, the word "tensor" also implies a set of rules on top of the pure data, but I guess it's fine to ignore this part and just call all multi-dimensional arrays "tensor". Wouldn't most people who aren't experts in mathematics think of tensors as essentially multi-dimensional arrays anyway? I know that's how I always thought of them.

I mean... C++ calls all array lists "vectors", whether or not they're members of a vector space.

[Neme12]

.NET's Vector<T>, Vector2 and Vector3 aren't strictly speaking vectors in the mathematical sense either, since not all of the rules for a vector space apply to them thanks to imprecise floating-point arithmetic or integer-overflow arithmetic (actually I'm not sure about this one, but anyway).

[tannergooding]

There is ultimately a balance required between consistency, simplicity, and various other considerations.

Most things in programming are not 1-to-1 with the math people learned in school and that's always been the case -- I could give hundreds of examples ranging from how tensor is used in other domains, to why lists are called std::vector in C++, to the fact that x + 1 > x does not always hold true, and so on. Put more simply, computer math is not real math, its a loose approximation.

I argued for SpanND, or a similar more general purpose name, and was outvoted by the other members of the core API review team, I'd rather not continue that discussion further as its unlikely to change. We just have to trust in the collective experience of the API review team, and the overall positive history of .NET APIs exposed in the last 20 years. Its rare we get something truly wrong.

Rank is because that's the near universal terminology across other ecosystems and is the terminology already used in .NET. It's prevalent enough that Wikipedia has Rank of a tensor disambiguated from Rank (linear algebra): https://en.wikipedia.org/wiki/Rank

Lengths was changed to Shape similarly because that is a more common terminology and there was already confusion points raised around Lengths. The shape of a vector, matrix, or tensor is simple the lengths of its dimensions, so 3x3 for example. A shape is often described using plural data and so it follows that it could return a span. API review hasn't gotten to the point of discussing this change yet, however, so its still to be determined if that is the better or worse name. We should get to it in the next review for this proposal.

[gaviny82]

This was discussed for a long time in the API review with @tannergooding constantly reiterating this point, but the decision has been made regardless, so I don't think it's worth discussing the names any further. It's true that the span isn't specific to "Tensor" and can represent any multi-dimensional arrays/data, however you can think of any multi-dimensional array as a tensor - if I oversimplify a little bit, I would say that "tensor" is just a mathematical way of saying multi-dimensional array. A programmer might not think of their bitmap 2d array as a tensor nor a matrix, but that's essentially what it is mathematically.

I'm happy with TensorSpan<T> if it's designed specifically for tensors, and we expect T to be constrained to numeric types that allow addition, multiplication, etc. SpanND<T> would mean a span more general than this, as T can be anything, like in Span<T>.

[Neme12]

API review hasn't gotten to the point of discussing this change yet, however, so its still to be determined if that is the better or worse name.

Right, as opposed to the names, I didn't see this part discussed at all and it surprised me it had just changed. If it's still to be discussed, I'm adding my argument for why "lengths" makes a lot more sense to me as someone not deeply familiar with the domain (which I guess is the same argument that Jeremy was making against the name SpanND - it's unclear what "ND" is to anyone not deeply familiar with the domain, like him or me), and who would have no idea what a "shape" is, whereas "lengths" is immediately clear to everyone. Although it's true that everyone not familiar with the domain will have to look up what strides are anyway - well, unless they just convert a multi-dimensional array to a TensorSpan and don't need to worry about strides.

[tannergooding]

I'm happy with TensorSpan if it's designed specifically for tensors, and we expect T to be constrained to numeric types that allow addition, multiplication, etc. SpanND would mean a span more general than this, as T can be anything, like in Span.

It is explicitly not constrained nor is Tensor<T> constrained. There are many scenarios where you want to have tensors of other data types, even things like bool or string in some ecosystems.

Instead, the APIs like Add or Min will be themselves constrained to only be available for types that support the interfaces. The plan is for cases like operator + to be done using the future language feature extension operators so that they can be similarly exposed only where it makes sense.

[Neme12]

What's the purpose of the Tensor<T> type anyway if it's just a multi-dimensional array though? Is it just that it can be used in a type-safe way regardless of how many dimensions it has? I had an idea for a (hopefully easy enough) solution to this here - just make all arrays of any dimension of T type derive from a new class Array<T> (which derives from Array) as an intermediate between the concrete array and Array, using the same runtime tricks that make arrays be derived from Array. I don't know a lot about how the runtime works so this might be a lot of work, but I would argue for this solution anyway as it would avoid bifurcating the ecosystem between people who use multi-dimensional arrays and people who use Tensor<T>. As @bartonjs said with regards to NSpan, "one person does a lot of work, and everyone benefits" - whereas if a completely separate type is added, it 1) adds bifurcation to the ecosystem, and 2) it will be a lot of work to plumb this through every API that currently uses multi-dimensional arrays. If that's really what Tensor<T> is - just a wrapper for type-safe multi-dimensional arrays, I would definitely argue for the Array<T> solution as opposed to adding a new, completely separate type. Then everyone can take their existing multi-dimensional array, and pass it to an API that takes Array<T>.

[tannergooding]

I'm adding my argument for why "lengths" makes a lot more sense to me as someone not deeply familiar with the domain

I definitely appreciate that. I was just giving the reasoning on why I had changed it and it being around the fact that people in API review had already expressed confusion around lengths, which had been my initial proposal.

Every type always has some learning curve and at least to me, its a question of how deep that curve is. I personally preferred SpanND because ND is easier to explain than Tensor. Almost all books out there (whether programming related or not) use the term n-dimensional when referring to an unknown number of dimensions. They similarly abbreviate things like 3-dimensional to 3D and so it follows that n-dimensional is shortened to ND. So, to me, both TensorSpan and SpanND had a learning curve, but one requires making a simple statement "n-dimensional", the other requires explaining what a tensor is and by extension requires talking about n-dimensional data and likely giving examples or context from other languages that have all unified on the term (python itself uses ndarray and even C++ has a proposal to expose a similar type).

Concepts like stride are similarly required to be explained, since it represents a multi-dimensional slice and therefore the data may not be entirely contiguous. That is the near universal term to represent the concept with some domains using more specific terms for specific dimensions (in image processing or games its often slice, pitch, width, and similar terms used).

Shape vs Lengths is going to be generally a similar consideration and with required discussion around which is better and represents the concept here more appropriately.

[gaviny82]

Rank is because that's the near universal terminology across other ecosystems and is the terminology already used in .NET. It's prevalent enough that Wikipedia has Rank of a tensor disambiguated from Rank (linear algebra): https://en.wikipedia.org/wiki/Rank

I agree "rank" is widely used and it is also the terminology used in math, but the problem is there's no alternative name for Rank (linear algebra). Not sure if there's any plan for a Matrix<T> type, but this is important if we want to build libraries based on the BCL to use .NET to be used for scientific computation like MATLAB and other languages.

Also, I don't think the System.Numerics.* should be designed for people who don't have domain knowledge at all. It is important to make these types friendly to beginners, but we should also consider professional uses in scientific computation. Personally, I hope I will be able to use C# instead of MATLAB or python for these applications in the future.

[tannergooding]

What's the purpose of the Tensor type anyway if it's just a multi-dimensional array though?

It basically is Array<T> and has the appropriate APIs to expose and operate with multi-dimensional data in a more type safe/beneficial way without breaking the existing ecosystem or making it non pay-for-play.

More importantly the general setup of APIs here will allow for some UnmanagedTensor<T> or similar in the future which is explicitly allowed to track native memory and which will require users to think a bit more carefully about ownership, but which then allows working with more than 2.14b elements in a single dimension (which GC allocated types don't allow today).

[Neme12]

I hope the Array<T> solution can be represented in API review/elsewhere, as it is the same argument that Jeremy was making against adding a separate "native span" and in favor of fixing the existing span. In this case - just fix the existing multi-dimensional arrays to derive from Array<T>. It can't be that much work I imagine? And this should be a lot easier than something like changing Span to use nint as this change wouldn't be breaking to anyone, or anyone I can think of, as the type doesn't currently exist so noone is currently using it.

[Neme12]

It really feels to me like the Tensor<T> class is just a workaround against having to do a few runtime changes to improve multi-dimensional arrays. But why not just improve the runtime, and everyone will benefit?

Or the Array<T> can be called Tensor<T> if we prefer - the main point is that all multi-dimensional arrays would automatically derive from it, just like they currently derive from Array, as an intermediate base type. Then people can use their existing multi-dimensional arrays and pass it to an API that takes an Array<T>/Tensor<T>/whatever API review would decide to name it. With no extra allocation. And people can use existing multi-dimensional array syntax to create these things.`

[gaviny82]

What's the purpose of the Tensor type anyway if it's just a multi-dimensional array though? Is it just that it can be used in a type-safe way regardless of how many dimensions it has?

I think it's just how every language or library used in machine learning does it. Other libraries may also need to define a type for tensors on GPUs. This can be done by implementing the ITensor interface, and a machine learning framework can treat the current design of Tensor<T> for CPU and the GPU tensors in a universal way.

[tannergooding]

Not sure if there's any plan for a Matrix type

There is not, that is covered by simply Tensor<T> currently.

but this is important if we want to build libraries based on the BCL to use .NET to be used for scientific computation like MATLAB and other languages.

It is impossible to come up with names that are non-conflicting. The entire scientific and mathematical domains have this problem where a single term is used to represent both loosely related and entirely unrelated concepts. It is something that such domain specific contexts need to rationalize themselves, such as by disambiguating if that's appropriate for them.

It is not worth making the thing that is shared across domains more verbose and less usable.

Also, I don't think the System.Numerics.* should be designed for people who don't have domain knowledge at all. It is important to make these types friendly to beginners, but we should also consider professional uses in scientific computation. Personally, I hope I will be able to use C# instead of MATLAB or python for these applications in the future.

As I've said, there is a balance between making things consistent, simple, approachable, etc. That applies both within the ecosystem itself (.NET) and outside the ecosystem (the rest of programming, scientific domains, math domains, etc), where consistency within the ecosystem is generally the most important.

[tannergooding]

It really feels to me like the Tensor class is just a workaround against having to do a few runtime changes to improve multi-dimensional arrays. But why not just improve the runtime, and everyone will benefit?

Because "just" is massively complex, potentially breaking, error prone, and doesn't really solve the problem across the considerations that are actually required.

As much as it would be nice if we could "just" fix Span or T[] to use nint, that's basically impossible at this point for all the reasons I detailed in the actual API review. As a summary, it is not pay for play and would regress perf for nearly every app in existence for something which is almost never needed. It introduces new arbitrary failure points for everything that hasn't updated to actually be nint aware yet (with no way to determine what is actually going to work or not).

You can't really extend System.Array for the same reasons, but there are even more complexities involved there that make this difficult. There's also the reasons I've gotten into on other threads that specifically pertain to doing large (85kb or greater) or huge (megabyte or larger) allocations in general and how those need to be very carefully handled/considered.

A new type and interface is the simplest/cleanest way to do this. The term tensor is being used because that's more consistent with other ecosystems and the scenarios where you typically need to work with such data. It not being in System also matches how often .NET developers (across the entire ecosystem) are going to use/need such types. There are specific apps, libraries and scenarios that will use these tons and so its beneficial to provide. But the vast majority of applications will never even know they exist (much as multi-dimensional arrays already are in .NET).

[Neme12]

Because "just" is massively complex, potentially breaking, error prone, and doesn't really solve the problem across the considerations that are actually required.

How would the idea of an intermediate base class between all arrays and System.Array be breaking? Such a type doesn't currently exist so noone is using it, so how could anyone be broken? This should be a much easier effort than changing Span to use nint, which would break every for loop that iterates over a span. This should break nobody, unless I'm missing something? And it's not like the idea that arrays would magically derive from something would be a whole new concept for the runtime - the runtime already does it with System.Array, and a whole bunch of interfaces as well, so it shouldn't be a huge amount of work to do it with an additional intermediate class as well.

I don't know why it would be error-prone or massively complex either, unless the runtime code that handles arrays deriving from System.Array and from IList<T>, IReadOnlyList<T> etc is written in such a bad way that it cannot be extended with just one additional class. Even though it was extended when IReadOnlyList<T> came out.

Anyway, I guess I said enough for this argument, thanks for the discussion, I just hope that the idea that such a change would be massively complex is really verified with the runtime folks. (Unless you work on the runtime as well in which case I apologize for not trusting your judgment enough, it just says you're on the .NET Libraries team in your profile. and from what I've seen in API review videos, runtime costs of various things are always determined by someone like Jan - e.g. "let's add these methods to the class itself unless Jan yells at us" 😄).

[tannergooding]

Such a thing doesn't provide any benefit and makes the general type system of .NET more complex with more considerations.

System.Array is itself limited to int in any single dimension, so you can't change it to (or have a derived type) now support nint without all the same considerations that Span<T> requires.

The runtime synthesizing interfaces and types behind the scenes requires languages to understand this and to specially handle that premise, that has cost. It also impacts the cost for dynamic type checks, interface casts, and general complexity of the ecosystem elsewhere. -- There's also, notably, a difference between a new interface and a new concrete base type in terms of considerations/complexity there.

Unless you work on the runtime as well in which case I apologize for not trusting your judgment enough, it just says you're on the .NET Libraries team in your profile. and from what I've seen in API review videos, runtime costs of various things are always determined by someone like Jan

I'm on the libraries team, but due to my areas of ownership in the libraries I'm fairly familiar with the runtime side of things and regularly contribute to the JIT (that's needed to provide the SIMD support for intrinsics or to version the primitive types for example). I do defer to people like Jan (the runtime architecture) and people who are actually on the JIT team in most cases, but this is one of the areas where it's been pretty heavily discussed and so I believe I'm in the generally correct area with the summary I've given here

[Neme12]

Fair enough. It just seems to me like if .NET were designed today, after generics and everything, multi-dimensional arrays would have been designed in a better way, with all of these use cases in mind, and I'm pretty sure someone even mentioned this in this API review - as deriving from some sort of MultidimensionalArray<T>, so that's why I'm more in favor of going that route.

[tannergooding]

Yes, I expect that if we were to design .NET from the ground up there would be a number of changes made to how things were defined/exposed.

But, we're on a 20+ year old ecosystem and so backwards compatiblity and not resetting existing source code and/or binaries fully to 0 is important.

[Neme12]

But, we're on a 20+ year old ecosystem and so backwards compatiblity and not resetting existing source code and/or binaries fully to 0 is important.

Yes, that's the argument against changing Span<T> from int to nint. I'm not making that argument though. Or is there an issue with my proposal that I missed that would break backwards compatibility in any way? I'm really confused, because your arguments seem like you're talking about the Span thing, not what I'm talking about, arrays.