Open cjnolet opened 1 year ago
I think that the matrix namespace should have functions that only use as input a matrix (and optionally a few parameters). So this rules matrix-vector operations out of this namespace. For element-wise operations, they can also be done on vectors so it would be non-natural to use the matrix namespace on a vector input. I also don't consider them as linear algebra so creating an additional namespace seems like a good option in my opinion.
The matrix-vector operations should go to linalg and be removed from matrix. For norm computation I'd tend towards the linalg namespace as well but I consider that having it under matrix is not totally wrong too. So there could be a wrapper in matrix for convenience.
There's a little bit of redundancy between linalg and matrix currently but it's unclear in some instances where some of these functions belong. For example, linalg contains several functions that aren't linear algebra, such as many element wise functions, yet since they are element-wise, these functions aren't quite matrix operations either. The matrix namespace also includes several matrix vector operations which I think be whether considered linear algebra ops, but do we at least keep a wrapper in the matrix namespace for convenience for so we consolidate these with linalg directly and remove them from matrix altogether?
The element-wise ops make me wonder if an additional namespace is needed. I think these just might require some more thought /planning to get right.
Another example- do things like matrix contractions and reductions such as norm computations belong in the matrix namespace or the linalg namespace? We are moving things like argmax/argmin and top k selection into matrix soon, does that mean all column-wise reductions should move to matrix?
In addition, there are matrix vector operations currently in both linalg and matrix. There's just been inconsistent use of the matrix and linalg namespaces across the board, so we should really think through when operations belong in one vs the other.