Similarities/Differences to the manifolds from Manifolds.jl

[kellertuer]

Hi, this looks like a super interesting package – are the manifolds then used within Tensorkit?

If so, would it maybe be useful to check whether we can join forces with the following idea: We have Stiefel and Grassmann (also generalises, complex and symplectic) variants in Manifolds.jl – so if (really just if) that would fit tensor kit as well, we could check how to maybe join forces here? We also provide different metrics and retractions.

Would be interesting to learn whether that is a thing one can do here (though that would diminish the purpose of the package itself here).

[Jutho]

Hi @kellertuer , thanks for your interest.

I agree that this package certainly has a similar goal as Manifolds.jl (but is of course much more limited). The idea is that TensorMap objects from TensorKit.jl are some kind of generalisation of matrices. In the widest sense, they are "homomorphisms between different objects in a Vect-enriched category", which means that they do behave as linear maps, but between vector spaces (or some generalisation thereof) that can have a lot more structure then just a total dimension. In particular, both the domain and the codomain of the linear map are assumed to have a tensor product structure, they are best thought of as not a single vector space but a tensor product of vector spaces. That is why these objects are more like multidimensional arrays than like matrices. However, there can be additional structure within these individual vector spaces, that makes that they cannot easily be represented as explicit arrays, and therefore they are completely disconnected from the AbstractArray type hierarchy and everything related to it.

However, such a TensorMap instance can indeed have the property that A' * A = (identity on domain), which I would call an isometric tensor, and thus corresponds to a point on a Stiefel manifold (or Grassmann if indeed there is an additional invariance with respect to basis changes on the domain side). This Stiefel manifold is thus specified by the codomain and the domain of these maps (which are represented by TensorKit.ProductSpace objects).

I am definitely aware of the existence of the ecosystem of Manifolds.jl and related packages, and might have looked at it for a brief time, but it wasn't clear to me whether this kind of generalisation (where the points on the manifolds are not AbstractArray objects, and the specification of the manifold needs a lot more structure) would be a natural fit. As such, I went on and developed my own implementation, which certainly also has a very different structure and interface, though I am not necessarily bound to that.

I am certainly open to discuss the possibilities for collaboration, as neither the manifold stuff but especially the different optimisation algorithms that are compatible with it are within my main expertise. Hence, if somehow what would be needed at our side could be supported from within Manifolds.jl, that would lighten the maintainance burden on our side, and immediately give access to many more optimisation algorithms as well potentially other manifolds that are useful in our applications.

For further reference: optimisation over these manifolds is done with https://github.com/Jutho/OptimKit.jl , and these manifolds are used, not in TensorKit.jl itself, but in the packages built on top of it like https://github.com/maartenvd/MPSKit.jl and https://github.com/mhauru/MERAKit.jl .

[kellertuer]

HI @Jutho, thanks for your detailed response. That sounds like all your entries are indeed multidimensional arrays (which might still allow them to also be points on a manifold – e.g. Stiefel as you write).

In Manifolds.jl we do not restrict types of points or manifolds – usually the manifold itself “decides” what it wants to accept (by multiple dispatch) and different representations of points (or phrased precisely different types) are possible for sure – for example we have 3 different point representations for hyperbolic https://juliamanifolds.github.io/Manifolds.jl/latest/manifolds/hyperbolic.html, and also the manifold itself can be any struct where you can store more information / structure.

Concerning Optimisation – my main field is therein as well – since the first thing I did start was Manopt.jl ( https://manoptjl.org/stable/ ) which is exactly also optimisation manifolds.

So the main point of this issue (maybe it is even more like a discussion, but these are not activated here), is whether we can maybe share certain aspects of these works / packages :) and sure, if you depend on ManifoldsBase.jl only (like Manoptjl does) you gain “free acess” to more manifolds, as well as having less work to maintain the manifolds. These were my main incentive to start this, definetly not to say you should not define your own Stiefel manifold ;) There are often good reasons to do so – but sometimes one might also find common interests and synergy effects.