ShapedArray type and/or protocol

[stephentyrone]

Add a ShapedArray type and/or protocol suitable for use as the common currency type for vectors / linear algebra / image processing / tensors, like ndarray is for numpy.

This should hew pretty closely to what S4TF already does with their ShapedArray type; it doesn't need to be identical, but we need to have a good reason for any changes.

[dabrahams]

@stephentyrone Are you familiar with MTL4?

[dastrobu]

A ndarray like type for swift would be great! Some features of NymPy's array would be great to see:

• contiguous and non contiguous array handling
• array slices with strides
• copy free array reshaping

I implemented a basic NdArray type for Swift a while a go, implementing theses features. It is currently more like a draft implementation than meant for production use.

Regarding linear algebra operations, I would recommend not to implement them in Swift and just call BLAS and LAPACK routines available on Linux platforms and from the Apple Accelerate framework on mac platforms. BLAS and LAPACK have been optimised for decades and there are many implementations available optimised for specific platforms or tuned for specific machines (ATLAS). As far as I know the Fortran implementations are still fastest and one should simply link theses implementations like NumPy and many other linear algebra packages do.

Not being too familiar with the ShapedArray type, I think it lacks support for non-contiguous arrays and array slices with strides.

[stephentyrone]

Not being too familiar with the ShapedArray type, I think it lacks support for non-contiguous arrays and array slices with strides.

ShapedArray in S4TF does not support strides arrays or slices, and only supports one dimension ordering. My intention for SN is to both of these things (ndarray is a good mental model).

Regarding linear algebra operations, I would recommend not to implement them in Swift and just call BLAS and LAPACK routines available on Linux platforms and from the Apple Accelerate framework on mac platforms.

My intention is to make BLAS and LAPACK available using the shaped array type, but it should also be possible to implement linear algebra--this is a great check to ensure that we have a sufficiently powerful API to describe new operations in the future. So I'll also be implementing some of the BLAS and LAPACK operations in a module as an easy way to flush out API limitations and optimization shortcomings.

[awav]

Hello, @stephentyrone! I thought you might be interested in this PR, https://github.com/tensorflow/swift-apis/pull/680. In it, I propose to define math protocols for shaped arrays in swift-apis. This is quite close to what you are doing with swift-numerics. Plus, I'm more than happy to help out with linear algebra implementation in swift-numerics.

[phewdry]

Hi Artem am wondering on current status for this issue?

Hello, @stephentyrone! I thought you might be interested in this PR, tensorflow/swift-apis#680. In it, I propose to define math protocols for shaped arrays in swift-apis. This is quite close to what you are doing with swift-numerics. Plus, I'm more than happy to help out with linear algebra implementation in swift-numerics.

[SharanSMenon]

Is this coming?

[Datamance]

Following this - it would be really nice to have a great NDArray-like API (and all the other numpy goodies) sitting on top of BLAS and LAPACK, but in the Swift Runtime. I think the closest thing out there is Surge but that uses Accelerate and OpenCV under the hood.

[dastrobu]

@Datamance note that Accelerate is using BLAS and LAPACK under the hood, so I think there is nothing wrong with that approach.

Surge does have quite good linear algebra support! Compared to numpy, Surge seams to be missing array slicing with strides. Having multiple views on the same data (different shape, different strides) is one of the great features of numpy I guess.

There is also Matft, which supports slicing and strides, but is not supporting generic elements. Then there is NdArray (written by me as mentioned earlier in https://github.com/apple/swift-numerics/issues/6#issuecomment-575887043) with focus on slices and strides while keeping generic elements. But currently, linear algebra support is very limited.

I guess there are many more Numpy'ish packages out there, all having some strengths and shortcomings. So having an official swift-numerics type would be great.

[superlopuh]

One thing that I would suggest is to have the interface be flexible enough to be compatible with TACO and the MLIR implementation. It really seems like a great abstraction over dense and sparse arrays, and when the MLIR project gets wider adoption it would be great to leverage all the optimisations that make more sense in the compiler than in framework code.

[superlopuh]

This is probably a better link to the TACO project: http://tensor-compiler.org

[JimWallace]

A noteworthy development in the Apple ecosystem: https://github.com/ml-explore/mlx

[dastrobu]

MLX is pretty nice, except that it has only a C++ and python interface, which makes it hard to use with Swift. Also it is only available on Apple Silicon chips AFAIK. This makes it hard to use for server side Swift applications.

[JimWallace]

MLX Swift: https://swift.org/blog/mlx-swift/

[stephentyrone]

MLX hews very closely to python/numpy-style for arrays, which is what the ML community is used to, but not necessarily what a more general-purpose library wants to have. There's lots of room for other work here.

[Lucca-mito]

And, crucially, MLX is Apple Silicon only.

[JimWallace]

MLX hews very closely to python/numpy-style for arrays, which is what the ML community is used to, but not necessarily what a more general-purpose library wants to have. There's lots of room for other work here.

Very fair.

This should hew pretty closely to what S4TF already does with their ShapedArray type; it doesn't need to be identical, but we need to have a good reason for any changes.

Maybe a good starting point would be looking at commonalities and differences between MLX and S4TF's ShapedArray? My sense is that there's some reasonable overlap already?

[wigging]

MLX hews very closely to python/numpy-style for arrays, which is what the ML community is used to, but not necessarily what a more general-purpose library wants to have. There's lots of room for other work here.

So what features should a general-purpose library have? And what improvements would such a library have over something like MLX Swift?

[Lucca-mito]

What improvements would such a library have over something like MLX Swift?

One potential advantage that such a library could have over MLX, NumPy, Matlab, etc. would be statically shape-safe matrices, unlocked by the recently-pitched integer generic parameters. If the library contains something like:

public struct Matrix<
    Element, let rowCount: Int, let columnCount: Int
>: RandomAccessCollection {…}

extension Matrix where Element: Numeric {
    public static func * <
        let lhsRowCount: Int, 
        let lhsColumnCount: Int, 
        let rhsColumnCount: Int
    > (
        lhs: Matrix<Element, lhsRowCount, lhsColumnCount>, 
        rhs: Matrix<Element, lhsColumnCount, rhsColumnCount>
    ) -> Matrix<Element, lhsRowCount, rhsColumnCount> {…}
}

extension Matrix: AdditiveArithmetic where Element: AdditiveArithmetic, rowCount == columnCount {
    public static func +(lhs: Matrix, rhs: Matrix) -> Matrix {…}
    public static var zero: Matrix {…}
}

extension Matrix: Numeric where Element: Numeric, rowCount == columnCount {}

Then the compiler would statically verify that all matrices being added or multiplied have the right shapes, and hovering on a matrix variable in VS Code (or option-clicking it in Xcode) would show the matrix's shape. Instantly knowing matrix shapes at code-writing time is certainly a feature I wish I had when I was writing NumPy and Matlab.