search

rapidsai / raft

RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.
https://docs.rapids.ai/api/raft/stable/
Apache License 2.0
722 stars 187 forks source link

[FEA] Consolidate some redundancies between matrix and linalg where it makes sense #871

Open cjnolet opened 1 year ago

cjnolet commented 1 year ago

There's a little bit of redundancy between linalg and matrix currently but it's unclear in some instances where some of these functions belong. For example, linalg contains several functions that aren't linear algebra, such as many element wise functions, yet since they are element-wise, these functions aren't quite matrix operations either. The matrix namespace also includes several matrix vector operations which I think be whether considered linear algebra ops, but do we at least keep a wrapper in the matrix namespace for convenience for so we consolidate these with linalg directly and remove them from matrix altogether?

The element-wise ops make me wonder if an additional namespace is needed. I think these just might require some more thought /planning to get right.

Another example- do things like matrix contractions and reductions such as norm computations belong in the matrix namespace or the linalg namespace? We are moving things like argmax/argmin and top k selection into matrix soon, does that mean all column-wise reductions should move to matrix?

In addition, there are matrix vector operations currently in both linalg and matrix. There's just been inconsistent use of the matrix and linalg namespaces across the board, so we should really think through when operations belong in one vs the other.

lowener commented 1 year ago

I think that the matrix namespace should have functions that only use as input a matrix (and optionally a few parameters). So this rules matrix-vector operations out of this namespace. For element-wise operations, they can also be done on vectors so it would be non-natural to use the matrix namespace on a vector input. I also don't consider them as linear algebra so creating an additional namespace seems like a good option in my opinion.

The matrix-vector operations should go to linalg and be removed from matrix. For norm computation I'd tend towards the linalg namespace as well but I consider that having it under matrix is not totally wrong too. So there could be a wrapper in matrix for convenience.