Replace Tensor<Double> with MatrixN

[marcrasi]

Matrix3x3 is faster because it will have no heap allocation and no TensorFlow overhead.

As mentioned in https://github.com/borglab/SwiftFusion/issues/42#issuecomment-627413240, this is currently the main bottleneck for Pose3Slam.

[ProfFan]

As #68 is merged, the next step would be making the generic version of MatrixN

[ProfFan]

@marcrasi Could you change the G2O Reader a bit and add the ability to read in the Pose3 dataset here? I tried and it seems to be a bit too convoluted for me...

[dellaert]

tl;dr As commented on in PR #68 I believe there is value in seeing a matrix as an array of column vectors. You can also see it as an array of co-vectors corresponding to rows, but that is less common. matvec is then taking the linear combination of the columns.

Long version:

Because I like types, I actually prefer to think about tensors, where each dimension can be "contravariant" or "covariant". A column vector is a 1D contravariant vector, and a row vector is a 1D covariant vector. Things like 3D-position etc are naturally "contra-variant": if you transform the coordinate frame points transform in the opposite way. A matrix as we know it has both a contravariant index i and a covariant index j, e.g., j=0 extracts the first column.

You can only "contract" - a generalization of dot product - covariant with contravariant dimensions. This is the "Einstein notation": dot = p_i q^i.

What is matrix multiplication then? Using Einstein notation, we have A^i_k B^k_j = C^i_j, and so we can see that A^i_k is a bunch of column vectors a^i, indexed by k, and B^k_j is a bunch of weight column vectors b^k, indexed by j.

In physics you can mix and match co- and contravariant indices, and the order does not matter as they have symbols associated. But in an implementation you do care about the order, and so if we just implement Matrix here then you have to choose: an array of column vectors, or an array of co-vectors. Because we typically think of vectors as column vectors, the former is more natural and agrees with common sense.

Note there is one place where co-vectors show up all the time: automatic differentiation :-) The tangent vectors are column vectors, but adjoints I believe are typically seen as co-vectors. I forget where I saw that.

I once created a c++ template class that statically type-checked contraction, but it was a bit unwieldy, so I'm not proposing we go there. It's useful in multi-view geometry. But let's at least know that we are taking a column-oriented approach (as is default in Matlab and Eigen, I think because of similar reasons).

[marcrasi]

@marcrasi Could you change the G2O Reader a bit and add the ability to read in the Pose3 dataset here? I tried and it seems to be a bit too convoluted for me...

Yeah!

Note there is one place where co-vectors show up all the time: automatic differentiation :-) The tangent vectors are column vectors, but adjoints I believe are typically seen as co-vectors. I forget where I saw that.

Interesting. We used to have separate TangentVector and CotangentVector types in Swift AD. The differentials dealt with TangentVector and the pullbacks dealt with CotangentVector. Then we removed the distinction because it just added more complexity with no benefit for any use cases that we had at the time. (gradient decent in flat euclidean spaces for deep learning)

[marcrasi]

Update on my progress: I've been thinking about vectors and matrices. I think it's actually more important to get these working well before getting the generic factor graph working. A lot of pieces of factor graphs are generic over vectors and matrices, so we should understand our vectors and matrices pretty well before we put them inside factor graphs.

I have some ideas how to express fixed-size vectors, and how to express fixed-size linear maps (aka matrices) as fixed-size collections of vectors. I think I'll be able to have a PR ready to discuss by this Friday's meeting.

In the meantime, Dave is working in parallel on the heterogeneous collection types that we are going to use to store the generic factor graphs.