Support navigating the API and migrating equivariant application to escnn

[Danfoa]

This is not a bug, however, since there are no discussions allowed for the repo (might consider enabling them?), I ask here. This is a call for aid in navigating the API for migrating a working application of equivariant learning, that is outside of the typical uses of ESCNN, but well within its capabilities.

Say I have a Trivial Principal Fiber Bundle $A = \mathbb{E_d} \times B$, where as principal fiber we have $\mathbb{E_d}$ usual Euclidean Group ($O(d) \times T_d$) and $B \in R^n$ the base space (a manifold with some symmetric properties).

In my application the bundle is trivial, meaning that $A$ is globally a product of two manifolds (a lie group and a manifold).

Note that because of triviality, I am not quite sure that fiber bundle formalism brings valuable mathematical machinery. Since we can work our way to understanding A as a product space. However, I am still becoming familiar with the theory, so there are high chances I am wrong.

I am interested in using ESCNN to define the symmetries of this space (to be exploited later in learning applications), but I am having some trouble finding my way.

Say the space $A$ has a known finite symmetry group $\mathcal{G}$, with known actions representations given by:

\rho_A(g) a =     
\begin{bmatrix}
    \rho_\mathbb{E_d}{g} & \boldsymbol{0}\\
    \boldsymbol{0}      & \rho_B{g}
\end{bmatrix} \begin{bmatrix}
    e\\
    b
\end{bmatrix} \quad | a \in A, e \in \mathbb{E_d}, b \in B, \forall g \in \mathcal{G}

Where $\rho_\mathbb{Ed}]$ is the representation of $\mathcal{G}$ in the fiber space, taking the form of a finite group of Euclidean isometries (translations, rotations, reflections), while $\rho\mathbb{B}$ is the representation of the group $\mathcal{G}$ in the base space, taking the form of a representation of a finite group, which dependent on the data I am assuming to be in $B$ (might be scalars, vectors, other fields) will have a "known" representation, but not necessarily a regular, permutation, irrep.

I have two questions that I want to solve before migrating stuff.

1. Will the Gspaces formalism offer some utility in handling trivial bundles? Do I really need a Gspace? Or should I instead construct the two representations of the group independently and define $\rho_A(g)$ as a sum of the fiber and base representations ($\mathbb{E}_d$, $B$)?.
2. I think I understand that if I have a data dependant representation of the finite group in $B$, to create the representation for my input data, I have to find the orthogonal transformation $Q$ to go from a say "permutation rep" to the desired one of my input data. Am I right? is there a way to define the representations directly by "inputting" the "known" rep, and then using the framework to compute $Q$? That is, to define a using a specific vector-space basis my group representation, and let the framework compute all the required transformations to obtain a "permutation rep" "decomposed rep" etc.

I understand the questions is long. Thanks for taking the time even to read. And thank you for providing this tool! I hope I can contribute to this project soon with some valuable additions/examples and tutorials of the likes of these .

[Gabri95]

Hi @Danfoa

Sorry, but I am not 100% sure I understood your problem.

Could you make the problem a bit more concrete? E.g. which manifold B do you have in mind? Also, if you want to use our library to build something like a Gauge CNN over a manifold, the fibers in the frame bundle are some subgroup of the general linear group, so there are should be no translations as far as I understand.

In any case, our library can support compact structure groups, so you can not directly encode translations in the fibers.

Moreover, it sounds like you want to encode separately some feature vector which transforms equivariantly to the transformations of the base space B. In our library, the base space B is the space convolution is performed over and this is what guarantees equivariance to B, if B is a group.

If 1) you can do convolution over B and 2) the fiber has compact structure group G (i.e. a subgroup of O(n) for some n), then you can use our library. You will still have to extend it a bit, since you need a GSpace encoding a trivial fiber bundle (our library only currently implements bundles of the form R^n \rtimes G, while you need something like R^n x G). This should just requiring a new subclass of GSpace and RdConv with minimal changes.

Let me know if this is indeed what you need

Best, Gabriele